Shareware Beach

Saturday, 18 December 2004

Backup Strategy to The Test

Filed under: Software Development — Jan @ 18:45

As tends to happen when my computer reaches a certain age, my two-year-old computer’s hard drive was getting a little full. Actually, I still had plenty of free disk space, it just wasn’t arranged conveniently. So I decided to partly repartition the hard drive last night. Of course, I started with making sure all backups were up to date. I made new backups of all the data on the disk. I decided the OS backups I already had were recent enough.

After clicking an incorrectly labeled button in the partitioning software, I conveniently wiped out the hard disks partition table. The button was labeled “cancel”, which means “abort, and revert to previous state”. It should have been labeled “screw you” instead. Inexperienced programmers often label buttons “OK” and “cancel”, even when things are not OK, or when the button does not revert the situation to the last known good state. But that’s a subject for another day.

With the partition software indicating the whole hard disk as unpartitioned, I decided to put my backup plan to the test rather than waste time trying to figure out how to cancel the partitioning. (Partitioning doesn’t take effect until the computer is rebooted, so theoretically I should have been able to cancel it.)

So last night I spent over an hour repartitioning the hard drive from scratch, and restoring all the OS images. Everything went smoothly, and I spent most of the time using the laptop to catch up on my reading. The OS backups were about half a year old, and didn’t include the Delphi 7 SP1 update, and the move from IE to Firefox. So I had to reinstall that this morning, plus copying all the data backups from DVD to the hard disk. All in all, everything went smoothly. I now have a clean PC, with plenty of free hard disk space, and I can be confident that in a real emergency, my backup plan works.

A good backup plan should allow you to recover from three situations. The most frequent problem is “operator malfunction”, also known as “shooting yourself in the foot with the Save or Delete button”. The second problem is hardware failure, usually the hard drive going for an early retirement. Finally, natural disasters like a thief making off with PC plus backup disks can really ruin your day. I will never forget the story I heard about the shareware author who diligently made his backups every day, only to lose his entire business when a burglar gave his desk an early spring cleaning.

A good backup plan also takes into account that some files are updated far more frequently than others, and that some files are far more valuable than others. Finally, “operator malfunction” can also affect the backups. Backups should be extremely convenient to make, so you won’t be lazy and skip any.

Making a backup is only convenient if it runs unattended. That means you’ll need a large storage device to make backups to. I use a DVD writer which can burn 4.5 GB to a single layer disk.

Backing up the whole hard disk every time doesn’t work, since I can’t fit my 80GB hard disk onto a 4.5 GB DVD. A backup of your complete disk is also not very useful when you need to recover from operator malfunction. If you back up your whole disk to a second hard disk, and the next day you find out that you deleted the wrong file last week, you won’t find it on the second disk either.

To be able to backup the disk in pieces, I use the following strategy. First, I partition the hard disk. I create one 8 GB partition for each operating system I want to install. Currently, my PC runs Windows ME (for testing), Windows 2000 (for development), and Windows XP (for testing and beta software). I back up these partitions only a few times per year. Before I do, I make sure it’s running smoothly and is squeaky clean (deleting the recycle bin, browser cache, etc.). I back these up to DVD-R and keep them forever. This way I can always go back to a previous known good configuration. Since Windows tends to slow down after installing and uninstalling lots of software, I often restore one of those backups when I’ve been testing beta software or engaged in other risky behavior. No registry first aid or Windows fixer tool can beat a clean backup. The backup software I use copies the whole partition, not just individual files.

The remainder of the disk is my big data partition. I never save any data on the OS partitions, so I can restore the OS without losing data. Different kinds of data are stored in separate folders, and backed up to DVD+RW separately. That way I don’t have to back up data I didn’t work with. E.g. after a day of programming, there’s no need to back up my digital photos. I don’t bother to compress the data when backing it up. That enables me to easily read the files from the DVD with the original software, if I ever have to recover from operator malfunction. I rotate multiple DVDs to make the backups, so I can easily go back a few days if I messed up and don’t discover it until after I’ve made a new backup. Since CDs and DVDs are not indestructible, I’ll also lose only one extra day should the DVD with the latest backup become unreadable when I need it. Don’t think this can’t happen. I’ve had several CD-Rs become unreadable already. Fortunately, none of them contained my only copy of the data.

Rotating removable media is good protection against hard disk and backup media failure, but is not sufficient against operator malfunction for files that are frequently modified. I never modify my digital photos (I create a new file when editing photos), but I modify my source code all the time when programming. And programming mistakes can go undiscovered for quite some time. So every time I release an update, I will make a copy of the whole source tree plus binary distributables. I keep these backups on the hard disk, and copy them to DVD as well. If I break something in my source code, even if it is a corner case that isn’t detected until several versions of the product later, I can always go back to a previous version and compare the source code.

With operator and device malfunction taken care of, the last step is to make sure you can recover from serious disasters. If your house burns down, that’s terrible. If you lose your shareware business, that’s terrible. Make sure you don’t lose your business when your house burns down. The only solution is a geographically separated backup. An easy way, what I do, is to regularly mail a DVD to my parents. All they need to do is keep it somewhere they can find it if it’s ever needed. As a side benefit, those DVDs, and CDs before it, are a nice time machine showing the growth of the data I’ve gathered and produced.

This backup strategy has saved me numerous times, both in case of operator malfunction like yesterday or hardware malfunction. Just two weeks ago, one of the two RAM modules in the laptop died. In the process, Windows crashed with a blue screen, damaging some vital system files preventing the computer from booting. After discovering the chip was bad (by running Memtest86 from a bootable Linux CD), I removed the chip, restored the OS partition from backup and was up and running in no time. Transcend seems to need more time to honor their lifetime warranty.

If you don’t feel so confident about your backup strategy that you’re willing to risk to wipe out your whole hard disk like I did yesterday, then you need to rethink that strategy right now. Tomorrow, your PC might have gone to greener pastures.


  1. Excellent article. I noticed you stayed away from mentioning any product by name though and wondered why.

    I’d strongly suggest using a source control package (Vault is even free for a single user) even if you work by yourself. It is so helpful and much easier to use.

    You didn’t say it explicitly, but I’m guessing you have the daily data backups automated right? I’ve been doing mine manually, but after reading this, I think I need to get a bunch of dvd rw’s and leave one in the drive so the automated backup can write to it without needing me. Then I just have to remember to change the dvd everyday or so and take one to the office every week.

    Comment by Bill Nalen — Saturday, 18 December 2004 @ 20:33

  2. You should look at using CVS or Subversion for managing your source files, even for a single developer it is worth setting up. It gives you more flexibility than backups if you want to roll back some piece of code.

    Comment by Tony Edgecombe — Saturday, 18 December 2004 @ 22:45

  3. I did not mention any product names, because I wanted to discuss backup strategy rather than recommend particular products.

    My backup process is fully automated, except for starting the backup. At the end of each day, I put a DVD+RW in the drive and start the preconfigured backup session. The backup then runs unattended.

    Comment by Jan Goyvaerts — Tuesday, 21 December 2004 @ 13:21

Sorry, the comment form is closed at this time.