There are several problems with trying to create a large filesystem to store important data:
- raid 5 isn’t expandable unless you find a harddrive that’s the same size as the others
- LVM (Linux Volume Management) isn’t fault tolerant by itself, so if you lose one drive, your data is lost.
One of the best solutions I’ve found in dealing with these problems involves combining LVM with the Linux software RAID. You can cut up your collection of harddrives into common sized partitions, raid them together into several RAID5 collections (or even RAID1 collections), and LVM the resulting RAIDs.
Example? You bet! I’m going to assume you’re already familliar with linux RAID and LVM enough to pull this off.
Lets start off with a handful of 250GB harddrives. 60GB would be a reasonable partition size for flexibility, since you have more choices in harddrives divisible by 60 than 50. You’ll have a little left over, which you can still use.
So 3 250 drives, each having 4 partitions of 60GB and 1 partition of 10GB. We’re going to have to make 5 linux raid arrays (going to assume you’re using /dev/sda, /dev/sdb, and /dev/sdc in the array):
md0 contains sda1, sdb1, sdc1 (each 60GB)
md1 contains sda2, sdb2, sdc2 (each 60GB)
md2 contains sda3, sdb3, sdc3 (each 60GB)
md3 contains sda4, sdb4, sdc4 (each 60GB)
md4 contains sda5, sdb5, sdc5 (each 10GB)
Now create an LVM that spans all 5 raid arrays. Now you can partition your LVM to your choosing!
Now what if you found a 120GB drive you wanted to add in to the array without having to backup your data and recreate from scratch? Split it up into 60GB raid partitions. Then you can use the linux raid tools to grow them into md0 and md1 as a 4 partition raid5:
md0 = sda1, sdb1, sdc1, sdd1
md1 = sda2, sdb2, sdc2, sdd2
followed by using the LVM tools to grow the LVM to the extra available space.
What happens if /dev/sda breaks? The LVM will still happily run: md0 as [xUUU], md1 as [xUUU], md2 as [xUU], md3 as [xUU], md4 as [xUU]. What if the last, 120GB drive breaks? Still protected! only md0 and md1 will have problems, the rest will show as up to date.
If you needed to add an 80GB hard drive, you’d have to split it into one 60GB partition, one 10GB partition, and 10GB of free space that you can use later if you find another harddrive size that has 10GB left over. You should not run two partitions of a single raid array on one drive (two 10GB partitions on one hard drive in one array), because if you lose that drive, that one array is lost. If you lose one array, your LVM is corrupt. If you add TWO 80GB drives instead, your problems go away. Assign the 60GB partition to one of the arrays using 60GB multiples, then create an additional mirrored (RAID1) array using 2 remaining 20GB partitions.
Enjoy never having to backup and reinstall to expand your fileserver, ever again!
