Category: 電腦 / Computers

Quick Script To Sort and Rename from EXIF Date

I wrote a quick bourne shell script to sort and rename the image files I recovered from my formatted memory stick. It requires the perl exifdump script that comes with the Image::Info module (not the python script by the same name). Since the last pre-format image I had was 8160, I started the recovered files at number 8161. The filenames will not be exactly right due to images that were deleted and retaken, but close enough.

The script was coded up quickly so there is no documentation or error checking.

#!/bin/sh
#

NUM=8161
PRE=DSC0
EXT=.JPG
TEMPLIST=`/usr/local/bin/mktemp /tmp/ds-rename.XXXXXX`
TEMPSORT=`/usr/local/bin/mktemp /tmp/ds-rename.XXXXXX`

for file in V*.[jJ][pP][gG]
do
echo "Processing ${file}..."
DATE=`/home/jlick/bin/exifdump ${file} 2> /dev/null | /bin/grep DateTimeOriginal | /bin/sed -e "s/.*DateTimeOriginal -> //"`
echo ${DATE} ${file} >> ${TEMPLIST}
done

echo "Sorting..."
/bin/sort ${TEMPLIST} > ${TEMPSORT}

for file in `/bin/sed -e "s/.* //" < ${TEMPSORT}`
do
echo "Renaming ${file} to ${PRE}${NUM}${EXT}..."
/bin/mv ${file} ${PRE}${NUM}${EXT}
NUM=`/bin/expr ${NUM} + 1`
done

/bin/rm -f ${TEMPLIST} ${TEMPSORT}

Recovering Digital Photos From A Formatted Memory Card

We won’t get into details. Let’s just say that somehow a memory stick in our camera got reformatted.

All the pictures are gone, right? Fortunately the answer is no. Most format processes only put a new filesystem on the memory card. The data for the photos is still there, there’s just no longer any information on where they are.

Fortunately image files are in standard, recognizable formats, so it is possible for a specialized program to search the raw data on the memory card and reconstruct most, if not all of the image files. The image files can then be copied onto your computer as good as new.

Things aren’t completely foolproof though. First and most important is to stop using the memory card immediately as soon as you realize you have mistakenly formatted the card or erased an important image. Since there is no longer any information on where the old pictures are stored, any new pictures taken will overwrite the old, deleted pictures. To be safe, take the memory card out of the camera, switch it to “read-only” if possible, then keep it in a safe place until you can use recovery software on it. If you have kept using the memory card, it’s still worth trying a recovery. You probably would get back less than if you stopped using it immediately though.

Second, in most cases you cannot get back the original filenames or creation dates if the card is formatted. (If on the other hand the file was deleted instead of formatted then you may be able to reconstruct the file names and creation dates too.) However image files usually have some embedded metadata in EXIF format in the file which contains further detail on the images. In my case I am able to see the date information, but the filename information is not there. So at least I will be able to name the files in order by the time the picture was taken.

Third, depending on how fragmented your memory card is, the recovery program may not be able to piece things together exactly, so it is possible you will not be able to get back 100% of your images automatically. Different software will have different results in how accurately they can reconstruct the files.

I tried a few different programs and found I had the most success with PhotoRescue. PhotoRescue comes in three different versions depending on if you need it to run completely automated, or whether you are more of an expert in reconstructing images. The automated method will usually work pretty well but it is not 100%.

I used PhotoRescue Advanced, which is their most sophisticated version, however it is also much more challenging to use. With this version you can get a few percent closer to 100% recovery of your images. When I tried a competing product, it found 530 images on my memory card. However after I looked closer I found that close to a hundred were recovered corrupt, truncated, or otherwise unusable.

In comparison, PhotoRescue Advanced found a more modest 482 images, however all but 8 of those were undamaged. If using an automated reconstruction, this is the extent of how much you could recover. However, using the advanced features of PhotoRescue Advanced, I was able to completely reconstruct all but one of the damaged images manually.

The next problem was that it recovered not only the images that were on the card when it was last formatted but also older images that I had already copied off onto the computer. I was able to use the thumbnails and EXIF date information to sort out the files I already had. At the end of it all I had 411 image files that otherwise would have been gone forever. This all had to be done manually though.

The remaining problem is that the files were recovered in the order they were found which is not exactly the same order as they were taken. As I mentioned previously, this information is stored in the EXIF data, however for 411 images it’s a bit too much to sort through by hand. Instead I will be writing a perl program to read in the EXIF data and sort the files by date and restore the file timestamps as well.

For now though, I at least have the satisfaction of having the 411 pictures back and safe.

UPDATE: Windows XP’s explorer has a column for ‘Date Picture Taken’ which can be added to the view when using ‘details’ view. This can help you sort if you want to do it manually.

UPDATE2: I wrote a quick script to sort and rename from EXIF date. PS: For those of you who don’t watch Law & Order SVU, this also means your naughty pictures are still there after being deleted or formatted.

Some More Random Solaris Notes

To enable Jumbo Frames for Intel Gigabit Ethernet cards, edit /kernel/drv/e1000g.conf and make the following change:

MaxFrameSize=3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3;

To share a zfs filesystem (e.g. /pool/user) over nfs:

zfs set sharenfs=on pool/user

Use “on” or “off” to toggle it on and off. If you want to set particular nfs share options you can use those options instead (on is equivalent to ‘rw’):

zfs set sharenfs=”rw=host.example.com” pool/user

Enabling Samba on recent Solaris releases

I recently upgraded two of my Solaris boxes to the latest Solaris 10 release and had some problems getting Samba working correctly again. Originally Samba on Solaris was started via the /etc/init.d/samba script and depended on whether the /etc/sfw/smb.conf configuration file existed. Then when the service framework came into use it was converted to a service, and you enabled it by running ‘svcadm enable samba’. All pretty straightforward so far.

In current Solaris releases there is a subtle change that may not be immediately obvious. For most Samba configurations you have two daemon processes running, smbd and nmbd. smbd is what actually does the file serving. If you only have smbd running you can still access files but you need to manually enter the filesystem path to get there. nmbd provides the netbios/wins service which is what announces on the LAN that the computer is part of a workgroup and what its name is. This is the piece that allows you to go to “My Network Places” or “View Workgroup Computers” and have stuff just automatically appear.

The change in the current Solaris versions is that while both daemons were controlled together in the past, they are now controlled as separate services. So now if you just do ‘svcadm enable samba’ it turns on only smbd and you only get file sharing that you can manually access. If you go to “My Network Places” or “View Workgroup Computers” you won’t see anything from that server. To enable nmbd also you need to also run ‘svcadm enable wins’.

Why would they do things this way? In more advanced network configurations you may have a domain controller that performs the netbios/wins functions. In that environment, separately running nmbd on a standalone server could cause confusion. Others may not want to run it because they don’t want an intruder to be able to easily discover shared filesystems. In any case, for most simple LAN based file service implementations, such as a home or small office network, you would probably want to run both services.

Youtube Popularity

I don’t really understand how videos manage to get popular on Youtube. At current count my videos have been watched a total of 7,293 times. I’ve made no effort to promote them and have only posted links on my blog and a couple of times on Forumosa. Three of the videos are of the English version video explaining the driver’s license test in Taiwan which have been up for a while and are actually useful, so I can understand why they’ve been viewed a lot, as they are actually useful.

But the most popular of my videos is a not terribly exciting video showing a bit of surveillance video from my restaurant during an earthquake. It has no editing, no sound, no titles, and to be honest the it’s far from the most interesting earthquake video on Youtube (see my favorites for some better ones). I don’t mean to sell it short; I do think it is interesting and worth watching. It’s just that I’m boggled that it currently has been watched 3,255 times. And all it shows is some ads and lights swinging gently for a minute.

The only external link that shows up other than my own stuff is a link from taggy.jp which seems to be some sort of Japanese version of Digg or Delicio.us. Apparently the video currently has 8 points there. Other than that the only thing it has going for it is a good subject and tags in both English and Chinese and a decent description (only English though; my Chinese doesn’t do well at long stuff). But still, 3,255 views. Wow!

twquake 1.5.1

We had a couple of quakes today, including one that woke me up at 6:54 am this morning:

M 5.0 46.0 km ENE of Yilan City 2007-11-08 06:54 #074 (1108065450074)
http://www.cwb.gov.tw/V5e/seismic/Data/quake/EE1108065450074.gif
http://www.cwb.gov.tw/V5e/seismic/Data/quake/EE1108065450074.txt

A smaller one this afternoon which I didn’t feel brought to my attention a few minor bugs that I hadn’t resolved in a while in my twquake perl script which fetches earthquake data from the Central Weather Bureau. That led me to other bugs and cosmetic issues and eventually spawned a new version of the script. Behold: twquake version 1.5.1. (I also noticed that I never posted version 1.5.0 which fixed a major bug. Well, actually the bug is in LWP::Simple but I had to convert to using LWP::UserAgent instead to avoid the bug.) If anyone actually uses this program besides me, please download the new version!

For those who are interested, I also run two mailing lists which send out alerts with the information on each earthquake reported in Taiwan. There’s an English version and a Chinese version:

Sign up for Taiwan earthquake alerts:
English: http://lists.jameslick.com/mailman/listinfo/twquake-alert-en
中文: http://lists.jameslick.com/mailman/listinfo/twquake-alert-zh

Quick Comparison Guide To Dr.Eye 8.0 Versions

Here’s a quick “Comparison Guide” to the Dr.Eye 8.0 packages available in Taiwan. The feature sets between the different packages has changed since 7.0 and as far as I can tell the feature comparison chart is only available in the manual, which doesn’t help you decide which version to buy.

There are at least four packaged versions of Dr.Eye 8.0 available in Taiwan:

Dr.Eye 8.0 Luxury (Orange Box TW$1090 MSRP)
Dr.Eye 8.0 Professional Upgrade (Blue Box TW$1090 MSRP)
Dr.Eye 8.0 Professional (Blue Box TW$1650 MSRP)
Dr.Eye 8.0 Professional 2-User Pack (Blue Box TW$3300 MSRP)

To find the MSRP (retail price) look under the bar code at the bottom right of the back of the box.

Unlike previous versions, the “Luxury” package no longer contains “Chinese Traditional to English” translation functions. It only contains “Chinese Traditional to/from Chinese Simplified”, “English to Chinese Traditional” and “English to Chinese Simplified”. Because of this, it is probably of limited use to most foreigners who will probably want to translate from Chinese more often than the reverse.

Instead, you will probably be better off with the Professional package which in addition to the above also supports “Chinese Traditional to English”, “Chinese Simplified to English” and “Japanese to/from Chinese Traditional”. The other major difference is that the Professional package also contains more dictionaries.

If you decide to get the Professional package, be careful you don’t get the upgrade version by mistake (unless you are actually upgrading). The easiest way to tell is to make sure the MSRP on the back of the package is TW$1650.

Dr.Eye 8.0 lists in the system requirements that it requires the Traditional Chinese version of XP/2003/Vista but it actually works fine on the English version of XP. (Presumably you would have to have the East Asian support added, but this is a standard feature of XP which can be added from the XP installation disk.) It also comes with the user interface in English, Traditional Chinese or Simplified Chinese all on the same disk.

One other potential catch if you have old computer gear: The package comes on DVD, not CD.

Note: Though the Professional ‘2-User Pack’ list price is exactly twice that of the single-user version, the street price is only about 50% more.

Wifi vs. Microwave Ovens

Many people know that 802.11b/g/n wifi can suffer from interference from microwave ovens. What doesn’t seem to be well known is that it only affects part of the wifi spectrum and can be worked around. Microwave ovens operate at 2.45ghz which roughly corresponds to “channel 9” on wifi gear. They also interfere on +/- 2 channels. So if you have problems with wifi dropping when a microwave oven is in use, simply avoid using channels 7-11 on your access point/router. I switched to channels 3 and 4 and no longer have an issue of interference.

Web application development (part 4)

Et voila! My first application is complete. Since the last post I extended the edit module so that it would put in blank boxes for hours the store is open but for which there isn’t data already stored. (Actually I just have it put in entry boxes for everything from 8a to 11pm for every day, even though my Xingtian Temple restaurant is only open 10a-10p and Qingcheng restaurant is only open from 10a-10:30p on Saturday & Sunday. I’ll have to add in more smarts later so it will look at what hours the store is actually supposed to be open for.)

I encountered another couple of difficulties here, namely that when you add new inputs you have to also have a hidden form value for the foreign key id for the main table. So in my case since the main table is Daily and the multiple data fields are hourly, I have to have a hidden form value for data[Hourly][index][daily_id].

When it was editing existing fields this wasn’t a problem since it would figure out the missing values based on data[Hourly][index][id]. However when you are adding new stuff it is unable to make the connection and so the data gets stored without the foreign key and doesn’t get associated correctly. You also have to do the correct magic to create a new object when saving new (as opposed to updated) data, but that’s straightforward from the add example I referenced in part 3.

I also completed the add controller and add view with the ability to add the daily and hourly data all on one screen. This was actually quite straightforward once the edit stuff was done. Again, the main hitch is making sure you get the hidden field for the foreign key id in there or it won’t go in the db correctly.

The final piece was to give me a scheduling view. Basically I set up a scheduler controller and view under dailies which would take a store number, number of weeks and ending date of the schedule analysis. The last two are optional and default to 1 week and the date of the latest daily in the db for the store, respectively. It then will make up an average of units sold for each hour on each day (i.e. each of the Tuesday’s 8:00 sales averaged, etc.) and based on that would give a number of hours to schedule each employee.

I can’t describe it in much more detail than that because while this sort of calculation is common in food & beverage operations, the details of the method SUBWAYâ„¢ uses is proprietary. I’m still tweaking the algorithm on how to do the scheduling, but right now it’s roughly based on the Subway method with a few tweaks to make it look better to me. I also need to make the tweak parameters configurable in the database instead of hard coded in the code, but that’ll have to wait a bit.

It looks like I can cut down my morning and afternoon scheduling a bit and actually could use a bit more people at lunchtime than I had thought. But also overall it looks like I should be able to save quite a bit on labor costs with a more efficient schedule while at the same time improving service levels during busy times.

There’s a few other things I need to do. For example, there’s a fair amount of code that I probably should have put in the controller that I put in the view instead. I think I reverted to non-MVC programming practices quite a bit. I think from the MVC philosophy you shouldn’t be doing much more than displaying stuff in the view. Anything where it is calculating stuff should be in the controller. I also need to figure out how to modularize the code better.

But, hey, it works. Woo woo!