iPodBackup

The Making Of

2009.02.23
Back to iPodBackup

I recently purchased an iPod, not only for the music, but also with the intention of using it to backup my files. I have about 12 GB of music and another 5 GB of miscellaneous files in my home directory, so the 30 GB (refurbished) model was a natural fit with room to expand.

iTunes makes synching my music collection extremely easy. But what about my files? I don't want to have to drag all my stuff over manually - that's a pain in the ass, and I'm likely to forget the various hidden ".something" config files and folders so often used by *NIX software. There are numerous backup utilities available for OS X, but which one is the right tool for this job?

Constraints

First I should specify the job itself. My goals are:

Options

Given the above constraints, I think the problem is well-defined enough that we can start looking at software. The ones I tried out are as follows:

Setup

Note: Make sure your iPod is set to be mounted on the desktop. You can change the iPod's behavior in iTunes.

Open RsyncX. I want to back up my home directory to a folder called "Backup" on my iPod, which I named "aPod." That means the destination path is /Volumes/aPod/Backup. I figure it would be safer using the absolute path for my home folder, so the source path would then be /Users/amake. But things aren't quite right:

Now things should be looking like this: RsyncX screenshot Click "Generate," and the command will be saved in your home folder as iPodBackup.command. Double-clicking on this file will execute it in the Terminal. You can also execute it from the command line by entering ~/iPodBackup.command.

Tweaking

But wait! There are a couple problems here: The first is that if you installed rsync along with RsyncX, calling it from the Terminal will execute the version that ships with OS X, /usr/bin/rsync. This version doesn't understand the resource fork, which is an important part of some Mac files. The version installed with RsyncX, on the other hand, does understand resource forks, and it is located in /usr/local/bin/rsync. Open up iPodBackup.command in a text editor, such as TextEdit, and modify it to use the "smart" version, like so:

/usr/local/bin/rsync -a -v "/Users/amake" /Volumes/aPod/Backup --eahfs --delete

Note: The source and destination paths must be in the order shown here. The arguments that begin with - and --, on the other hand, can be in any order. This isn't necessarily true of all commands, though, so be careful.

I'll take a moment here to explain the various arguments of this command:

The second problem is that this command is still going to copy all of my home folder, including ~/Music and various caches. The solution to that, it turns out, is in the manual for rsync, accesible in the Terminal by entering man rsync.

The answer is the --exclude option. Adding the argument --exclude "Music/" to the command will exclude any directores titled "Music," which is exactly what I'm looking for. You can repeat the argument as many times as you need to weed out other directories. Let's throw in an --exclude "Cache/" too:

/usr/local/bin/rsync -av "/Users/amake" /Volumes/aPod/Backup \
	--eahfs --delete --exclude "Music/" --exclude "Cache/"

Fancying it up

Now the command is complete! However, when you open it it will launch the Terminal, and when it's done processing you'll have to quit the Terminal. Plus, as it's not a true application, we're limited in our use of it in some ways. Luckily, there is a better way. Download Sveinbjorn Thordarson's Platypus, and we'll turn the command into a regular OS X application.

Platypus works most easily with shell scripts, so we're going to have to turn our .command file into a shell script. All you need to do is add #!/bin/sh to a new line at the beginning of the file, and for clarity's sake save the file as iPodBackup.sh.

Now we're going to add a few bells and whistles: As it is now, every time we want to exclude something new from the backup process, we'd have to start over from scratch. However, if we replace the --exclude arguments with --exclude-from file we can exclude files and folders based on a list stored in a file that we can easily edit. Let's call the file exclude.txt and the contents will be the things we want to exclude from the backup, each on its own line. (Lines beginning with # are ignored; enter man rsync in the Terminal, or see an online copy of the manual for a more in-depth discussion of the syntax of these rules.)

#exclude.txt
#This document specifies which directories NOT to back up to the iPod

Music/
Cache/
Caches/

The final product will be an OS X program bundle, i.e. a folder with a .app extension that contains all of the files necessary for the app to operate. Apparently the location of the bundle gets passed to the program as a variable, so exclude.txt will be inside the folder $1/Contents/Resources. We need to replace the --exclude arguments like so:

#!/bin/sh

/usr/local/bin/rsync -av "/Users/amake" /Volumes/aPod/Backup \
	--eahfs --delete --exclude-from $1/Contents/Resources/exclude.txt

Another bit of hoodoo we'll work here is a log to keep track of what exactly is going on. First we'll set the location of the log with LOGFILE="$HOME"/Library/Logs/iPodBackup.log. Now we want to use redirects to put data into the logfile. > will take the output of a command and create a file of it (careful! It can also overwrite existing files). >> will append the output of a command to a file. echo string will output a string of your choice, and date will output the current date. Also, we can generalize the source directory (my home folder) to "$HOME". Put all of this together for:

#!/bin/sh

LOGFILE="$HOME"/Library/Logs/iPodBackup.log
echo "Synching home directory to iPod..." > "$LOGFILE"
date >> "$LOGFILE"
/usr/local/bin/rsync -av "$HOME"/ /Volumes/aPod/Backup \
	--eahfs --delete --exclude-from $1/Contents/Resources/exclude.txt >> "$LOGFILE"

Note: LOGFILE is the name of a variable. To use the value of the variable you have to reference $LOGFILE. This is true for all variables.

Note: The double quotations around the variables are necessary to handle paths with spaces or other weird characters in them.

Note: The final line of the file begins with /usr/local/bin/rsync. Yes, all of that is one big honkin' command, so make sure it's all on the same line.

Now there's just one problem left: This only works with my iPod. If your iPod is named something other than "aPod," this script will not work. Unfortunately, there is no super easy way to tell if an iPod is connected or not, or what its name is. We can make a good guess, though, by using the find command to look for an invisible folder called iPod_Control that is present, to the best of my knowledge, on every iPod. This folder probably contains information used by iTunes and the iPod OS. We're not going to mess with the contents of this; we're just using the existence of the folder as a probable indicator of an iPod.

The appropriate command here would be find /Volumes -name "iPod_Control" -maxdepth 2.

This command will return, in my case, /Volumes/aPod/iPod_Backup. But that's only if my iPod is mounted. We should plan for the possibility that the iPod isn't mounted at all, in which case we don't want the backup to proceed. The answer is an if statement. We're going to save the result of the search as the variable DESTINATION, and we're going to test -z "$DESTINATION". In other words, we're testing whether or not DESTINATION has a length of zero, which would be the case if no matches are found.

DESTINATION=`find /Volumes -name "iPod_Control" -maxdepth 2`
if [ -z "$DESTINATION" ]
then
	echo "There is no iPod!"
	exit 1
else
	# Proceed with the backup
fi

Note: The ` marks are required so that the find command is evaluated. Otherwise the value of DESTINATION will be set to find /Volumes ....

This conditional statement ensures that the script will exit if the folder iPod_Control isn't found. In order to turn DESTINATION into the correct path for the backup folder, we just need to append /../, which sends us backwards one level, i.e. to /Volumes/aPod, and then the name of the backup folder, Backup.

DESTINATION=`find /Volumes -name "iPod_Control" -maxdepth 2`
if [ -z "$DESTINATION" ]
then
	echo "There is no iPod!"
	exit 1
else
	DESTINATION="$DESTINATION"/../Backup
	# Proceed with the backup
fi

Finally, we need to incorporate this logic into the rest of our script. Make sure to change the destination path of the rsync command to "$DESTINATION". While we're at it, we'll spruce up the info heading to the log a little bit, and add some explanatory comments.

#!/bin/sh

# Set the log file location to the appropriate place.
LOGFILE="$HOME"/Library/Logs/iPodBackup.log

# Test to see if an iPod is connected.  If yes, proceed.  If not, quit.
DESTINATION=`find /Volumes -name "iPod_Control" -maxdepth 2`
if [ -z "$DESTINATION" ]
then
	echo "There is no iPod!" > "$LOGFILE"
	date >> "$LOGFILE"
	exit 1
else
	DESTINATION="$DESTINATION"/../Backup
	echo "Synching \"$HOME\" to \"$DESTINATION\"" > "$LOGFILE"
	date >> "$LOGFILE"
	/usr/local/bin/rsync -av "$HOME"/ "$DESTINATION" \
		--eahfs --delete \
		--exclude-from $1/Contents/Resources/exclude.txt >> "$LOGFILE"
fi

If you've installed the RsyncX package, the above script is all you need. However, for the purposes of a distributable application capable of running on any OS X installation, we're going to go one step further. We need to package rsync itself within the application. That means we need to change its path to what it will be inside the .app bundle: $1/Contents/Resources/rsync.

Also, I decided that "Backup" is not that great a place to put things. If you have a copy of OS X installed on your iPod, you'd probably want the backup to go to the proper home folder location, which is /Users/$USER on the startup disk. Plus, this way you can back up multiple home folders to the iPod. So, the destination path should now be DESTINATION="$DESTINATION"/../Users/"$USER".

New in version 1.0.1: It turns out that rsync does not create the "Users" folder automatically, so we need to make it if it doesn't already exist. mkdir "$DESTINATION"/../Users should do the trick. Also, I made some minor changes to the logging system.

New in version 1.0.2: By default, "Users" folder is created with permissions such that other users can't write to it. To fix that, the folder is now created with permissions set to 777 (everyone can read, write, and execute) by adding the argument -m 777 to mkdir.

New in version 1.0.3: ~/Library/Logs does not exist for a newly created user. iPodBackup now creates it with mkdir -m 700 "$HOME"/Library/Logs if it doesn't already exist.

New in version 1.0.4: So that multiple iPods don't throw things off, we'll pipe the find command through head -n 1. This cuts out everything but the first line returned by find.

#!/bin/sh

# iPodBackup version 1.0.4 (2005-01-09) by amake

# Set the log file location to the appropriate place.
mkdir -m 700 "$HOME"/Library/Logs
LOGFILE="$HOME"/Library/Logs/iPodBackup.log

date > "$LOGFILE"

# Test to see if an iPod is connected.  If yes, proceed.  If not, quit.
DESTINATION=`find /Volumes -name "iPod_Control" -maxdepth 2 | head -n 1`
if [ -z "$DESTINATION" ]
then
	echo "There is no iPod!" >> "$LOGFILE"
	exit 1
else
	mkdir -m 777 "$DESTINATION"/../Users
	DESTINATION="$DESTINATION"/../Users/"$USER"
	echo "Synching \"$HOME\" to \"$DESTINATION\"" >> "$LOGFILE"
	$1/Contents/Resources/rsync -av "$HOME"/ "$DESTINATION" \
		--eahfs --delete \
		--exclude-from $1/Contents/Resources/exclude.txt >> "$LOGFILE"
	echo "Sync completed successfully" >> "$LOGFILE"
fi

Note: Do not copy and paste the above into your own script! I tried that, and the resulting file was completely fubar'd. This script doesn't erase anything, but if it goes haywire it could fill up your hard drive and make files in weird places. So instead of copying and pasting, download the script file if you're interested.

Note: As of version 1.0.1, iPodBackup does exactly what I originally intended for it to do. Versions higher than 1.0.x will focus on additional features. These features require significantly more complex code, so I won't be updating this walkthrough anymore. If, however, significant problems are found with 1.0.x, I will correct them in this walkthrough. I am no longer distributing 1.0.x, though.

Finishing touches

Again, assuming you've installed the RsyncX package, enter in the Terminal cp /usr/local/bin/rsync ~/Desktop to copy rsync to the desktop in preparation for inclusion in the app. Now we're ready to throw this mess through Platypus. Set it up as follows:

Platypus screenshot

I like turning on "Require Administrator privileges" because I can prevent the sync from even starting, if I want to, by clicking the "Cancel" button on the authorization dialog. Now create the thing and put it somewhere smart, like on your iPod.

But wait! We never addressed the automation issue! Well, thanks to a very nifty shareware (US$6.95) app called Peripheral Vision, by Granted Software, this is a ridiculously easy matter. After registering, you can set Peripheral Vision to launch an app upon insertion of a device or mounting of a volume:

Peripheral Vision screenshot

After the app launches, you will be prompted for your password. Enter it, and then open up iPodBackup.log to see the action.

You can also accomplish this kind of automation with iPod Launcher (US$4.95), or Do Something When (free). I have not used them, though, so I can't offer assistance with setting them up.

Note: The first time you backup it will take significantly longer than subsequent backups, since we've set rsync to only copy updated files.

If you'd like to adjust the exclude list, control-click on the iPodBackup app and choose "show package contents." exclude.txt is inside iPodBackup.app/Contents/Resources. You can also specify inclusions in this file; see man rsync for details.

And that's all she wrote. The script is completely general, except perhaps for the name of the target folder on the iPod. I don't think I can easily let the user choose that while retaining the automatic nature of the script, so I'm not going to bother.

Back to iPodBackup