I spent a little bit of time today looking into some of the storage solution choices out there.  I was mainly concerned with systems supported under Linux, but I didn't limit myself to just those.

My quest started this morning when I read a post on Gizmodo: Buffalo DriveStation: Serial ATA, Fanless Design. I had heard of Buffalo Technology before, I've often considered buying one of their products, or something like it.  What could be better than a 2 Terabyte box that you just plug into your network and configure?  Well...The Perfect Storage Solution of course!

A few simple use cases may describe what I'm looking for.  First, if a drive fails, I want to be able to replace it with no downtime, loss of data, and using any drive of sufficient size (at least as large as the one that failed) that I have on hand.  Second, if I'm running out of free space, I want to be able to add a brand new drive and start using it.  Third, if there is no more physical room for a new drive, I want to be able to migrate data off of the one of the drives (probably the smallest/oldest one), to make room for a newer, larger drive.  I suppose this is a direct consequence of satisfaction of the first requirement.

Out of these simple cases, I can distill a few must-have features:

• Scalable! I should be able to add more space to this thing with a minimum of hassle.  I consider having to unmount a filesystem, grow the partition, then grow the filesystem a hassle.  I also consider having to get drives of exactly the same size as the drives already in there a hassle too.  I want to add more space, not replace the space I have!  LVM comes close to achieving this.
  
• Fault tolerant: The Perfect Storage Solution should be able to handle at least one of the drives failing.  Better would be the ability to handle n drive failures.  RAID-5/RAID-6 work well here, but fail the scalability requirement.
• Cheap!  I shouldn't need proprietary hardware/software for this.  I should just be able to add another drive to my enclosure and start using the space.  Or at the worst, buy another enclosure and add the new drive to the new enclosure :)
  

Some other nice features would be things like:

• Snapshot support: great for backups, or when doing some kinds of admin work.
• Very large filesystem support.  It's very easy to get more than a few terabytes of data (legitimately!) these days :)
  

And as long as I'm writing a wish list:

• Transparent compression
• Transparent encryption
• Clustering (create one big pool of storage from drives scattered over a network)
  

So what's wrong with the devices like the one above?  Basically they don't scale well.  There's no nice way to integrate these things into one big pool of disk space.  You need to mess around with mount points, and put symlinks all over the place...unless you use LVM.

What's wrong with LVM?  It's not fault tolerant.  Sure, you can run LVM on top of a bunch of RAID devices.  But that means to add more storage in a fault tolerant way, you need to add a whole new RAID array since it's not really possible (as far as I know, somebody please correct me on this if I'm wrong!) to add a single drive into an existing RAID array.

Wikipedia's RAID page mentions the idea of a "write hole", and refers the reader to Jeff Bonwick's post on RAID-Z. The concept of the "write hole" does make some sense to me; basically if the drives lose power or crash while writing the parity data, then the data blocks and parity blocks may be inconsistent. It's not clear to me how RAID-Z solves this, and why you can't check those blocks when you restore power (especially when using some kind of journaled filesystem...although I suppose that the journal's parity data may have been corrupted as well!), but certainly data integrity is an issue that the Perfect Storage Solution must address!

I thought Sun's ZFS was promising for a little while.  But it seems that it doesn't handle drives of different sizes any better than RAID-5/6 does.  It's also unlikely that it will ever be available to Linux users because of licensing issues.

It certainly seems like LVM is the closest to what I'm looking for.  If only it managed parity data across physical blocks!  Then just polish up the ext3 online resizing functionality and life would be great!  Or maybe reiserfs/jfs/xfs would work better for a resizable filesystem?

Technorati Tags: linux, software, blog, storage