Randomize

For the second time in the past few weeks, I’ve heard of a photographer who lost many years’ worth of work due to their computer and drives being stolen. This has caused me to start re-evaluating my own backup strategy, and I thought I would share a few notes about what I’ve already learned and how I’m planning to improve my own data security.

IMHO, a good backup strategy involves five prongs: good drives, local live backup, local online backup, remote backup, and portable backup.

Good Drives

The first line of defense is that you should entrust your RAW and PSD files only to drives with a strong record of low failure rates. Based on recent numbers from a study done by Backblaze, Hitachi drives outperform other brands.

Of course, the best drives are solid state drives — their Annual Failure Rate (AFR) runs closer to 1%, while physical drives run 3-8% depending on their age. But for now, solid state drives are too expensive for serious photographers — we churn through way too much storage.

Live Backup

Assuming you aren’t reading this in the future where 4TB SSD drives have a 0.5% AFR and cost $100, you’ll still be using good old spinning-plate drives. The catastrophic death of these drives is not a matter of “if,” but of “when.”

So, the second prong in a good backup strategy is that photography drive volumes should always be created in pairs — a RAID 1 set (mirrored). With this approach, every byte written to one drive is written to both simultaneously, and both drives can be used independently if one of them fails.

In years past, RAID 5 was the gold standard, since it offered a better balance of usable space than RAID 1 (²⁄₃ of the space is available, versus ¹⁄₂ for RAID 1). However, as the storage size of drives have increased exponentially, so have the chances that when a drive in a RAID 5 set fails, that an unrecoverable error will occur during the rebuild, which then puts the entire volume in peril.

RAID 5 also relies on _proprietary_ logic that determines how the data and parity stripes are laid out on the physical drives. Thus, if the RAID _controller_ hardware fails and you can’t find replacement hardware that uses the same firmware, there’s a good chance your array will become an instant doorstop.

At the time of this writing, Hitachi 4TB drives run are $180, and cheaper drives are $140. If your average shoot runs around 16GB, you’ll be paying around $1.50 per photo shoot for storage.

Local Online Backup

A mirrored volume is not, by itself, a sufficient backup. It mitigates the issue of drive failure, but does nothing to protect you from yourself. If you accidentally trash the wrong folder, run a bad script, or overwrite the wrong file, you can lose hours of work, entire shoots, or in the worst case, everything.

The reason that a local online (i.e., connected and turned on at all times) backup is important is that, being human, you will forget to connect and use your backup drives.

I recommend software such as Apple’s Time Machine, which silently, reliably, and quickly backs up all changes you make to your files each hour, and makes it a cinch to restore the files, should the need arise.

It is preferable to locate this drive in your house, but in a a separate physical location from your main computer. This reduces the chances that a localized fire or theft will result in the loss of both your primary and backup drives.

If you’re an Apple fan, a good solution is to use an external hard drive connected to your Airport Extreme router. Apple sells a version of the router with a drive built in (“Time Capsule”), but as usual with Apple, the price is much higher than just connecting your own drive via its USB port.

These backups should contain not just your photographs, but also your boot drive, applications, Lightroom catalogs, and anything else you need to back up on your computer.

Again, because drives do fail and backups are difficult to restart from scratch, having a RAID 1 volume for your backup drive is a good idea. Unfortunately, the Airport Extreme does not support software RAID, so to use it, you need a drive enclosure with a built-in hardware RAID controller (so the Airport Extreme only sees one virtual drive).

I use an Akitio Hydra enclosure, which supports RAID 1 or 5 for up to 4 drives. I recommend against the Drobo — while I’m sure they’ve put a lot of work into their “BeyondRAID” algorithm, the fact remains that it is proprietary, and I’ve heard a number of horror stories of people losing Drobo volumes and having no means of recovering them.

Remote Backup

Having all of your primary and backup drives in one physical location (and online) is a bad idea, because it exposes you to a number of potential threats — a thorough thief, fire, flood, lightning strike, power surge, etc.

So, it is essential to have a backup in another physical location, and to keep it up to date. This is the piece I’m lacking in my own strategy, and it’s something I’m working to address.

The simplest solution is to ask a friend or family member to hold your backup drive for you, and swap them out once every few months (a safety deposit box would also work). If you don’t work from home, you could also just keep the drive in your office.

To make these remote backups, you may need to buy some additional software that can do incremental copies of your main drive to your backup drives — synching only the files that have changed. If you’re handy with the command line, rsync on OS X and xcopy on Windows can do this for free.

In this situation, encrypting the drive is probably a good idea, especially if your photographs are sensitive in nature (boudoir, art nudes, etc.). Even if you trust the person holding the drives, you can’t trust a thief who might take off with your drive while robbing their house. Fortunately, this is very simple to do in Disk Utility on a Mac, and using BitLocker on Windows.

There are a number of “cloud” backup services (Carbonite, Backblaze, Crashplan, Amazon Cloud Drive, Microsoft Drive, and Mozy, to name a few). On the plus side, these offer continuous (daily) backups and can allow you to access the files online from another computer. However, there are some disadvantages:

• They can be expensive over time compared to just buying a hard drive or two.
• You have to understand which of their plans you need to use. For example, if you use Carbonite, you would need to use the $100/year plan, not the $50/year one, because only the more expensive plan will back up something other than your main user directory (which will almost certainly not be the drive you’re using for your RAW and PSD files).
• Upload speeds can be terrible. Most Internet providers give you only a modest upload speed — mine is 1.5Mbps, which is 1/10th of the download speed. At this speed, sending 16GB of RAW files to the cloud would take a full 24 hours and would saturate my uplink, which might cause issues with other Internet usage. So before you consider a cloud solution, test your upload speed and do the math!
• Consider the risks of systems that don’t offer end-to-end encryption. Some services encrypt your files on their server and in transit, but they hold the keys, so anyone who compromises their system can read your files. The only safe encryption is where the encryption key never leaves your computer. If you don’t want your boudoir clients or models being involved in the next “The Fappening”-style breach, be sure you understand the basics of encryption and how they treat your files (good rule of thumb: if you can log into their site using a normal web browser and see your files, any “encryption” they say they do on your files is not sufficient).

I will say that of the cloud services I’ve seen, the one I like best is Crash Plan’s “Offsite Drive” option. This is a free service, it basically allows you to make a backup external drive, trade drives with your friend, and your changed files will be sent to your drive on their computer, directly and automatically, over the Internet (and vice versa). The drives are heavily encrypted (the right way), so you can’t see each other’s files. And if you want them to additionally store the files on their servers, they are happy to do that as well (for a fee of course).

While this concept from Crashplan doesn’t completely overcome the issue of upload speed, at least you are only uploading new/changed files to one another, not trying to upload your entire library of files. If you were trying to, say, upload 2TB of photos via a 1.5Mbps uplink, it would take over _4 months_ to complete the backup, so trading drives with a friend is far better than using a traditional cloud backup service, which is optimized for “normal” people who may only have 20-50GB of total data.

Portable Backup

The final piece to the puzzle is to have an emergency backup of your most important documents on your person at all times.

If the nightmare scenario happened and someone was able to compromise and destroy your files both on your local copies and your cloud backup, the goal of this backup would be to save (a) personal files of great importance, such as family photos and tax records, and (b) your _legacy_ of work as a photographer.

Carrying around multi-terabyte hard drives is obviously not an option (yet), but you don’t really need to. Right now, a 256GB flash drive runs around $70. This won’t be nearly enough for your RAW and PSD files, but you can at least use it to store a very large number of _full-resolution, final_ JPEGs of your work.

If you ever found yourself with only that flash drive remaining, the loss of the PSDs and original files would be regrettable, but you would still have a digital master that is suitable for making new prints.

Again, encryption is absolutely essential for this — if your USB drive is ever lost or stolen, you don’t want your personal information to be available to whoever ends up with the drive.

A good strategy is to create two partitions — one very small, unencrypted FAT partition with just a “readme.txt” file containing your contact information and a promise of a few bucks to return of the drive, and the second one for your main encrypted storage. Giving the smaller partition some extra breathing room (say, 4-8GB) might also be useful for keeping some basic data rescue programs, or just so you can use the drive in untrusted computers for short-term file transfers.

This final layer of protection may seem as if it borders on paranoia, but keep in mind that if every other backup you have happens because of automatic processes, you need at least one backup that requires a manual copy process.

Keep in mind, however, that current flash drive technology requires that drives be _used_ — if you let a flash drive sit dormant long enough (a year or two), you could end up with corrupted data. As such, flash drives aren’t a perfect replacement for other backup media. (SSD drives have the same bit-rot issue.)

Final Thoughts

Data is fragile, and thinking through the potential points of failure requires good planning and a solid basic understanding of basic technology. No one ever thinks something bad will happen to their data, until it does. Years or even decades of work can disappear in the blink of an eye. So, stop reading this and GO BACK UP RIGHT NOW. 🙂