Converting a Linux System to Raid1
One of the machines I manage was running Linux on a Compaq Proliant server with a SMART Array (6042) controller and two external cabinets. The system booted off of a SCSI disk on the built-in Adaptec controller and the disks in the external cabinets were configured with one disk per logical disks.
Why this machine had a SMART Array (i.e. Hardware Raid)
controller, is a tribute to public
purchasing rules and quite another story. The initial hardware
situation looked like this:
What was desired was complete redundancy from the SCSI controller to the cabinets. This is the kind of hardening of the system that was deemed appropriate for this system. The desired configuration was this:
The system used LVM volumes, to allow the database file systems to grow if desired. The final system would be made up of:
Migrating Volume Groups to Software Raid
Implementing Volume Groups on software raid is quite straightforward. Since I was using identical disks, things are easy. To accomplish this, you need a copy of the Linux Raidtools installed. The steps are:
In my situation, half of the disks were already in volume groups with data on them. Instead of the disks being partitioned, the volume group used the entire disk unpartitioned. This meant that I had to copy the data (I used the other disks, as there was enough free space), dismount the current volume, remove the current logical volume with lvremove, deactivate the volume group (vgchange -a n vgXX) and vgremove the volume group. I did the migration over two days, changing the mount points and leaving the mirrored LVMs when done.
Migrating the Boot Disk
The disk containing /, /usr etc. had no partitions in LVM volumes. I decided to keep it that way and just mirror the disk. Being conservative, I had a spare disk, which allowed me to move all my filesystems to the mirror pair and have my original boot disk just in case. You can migrate an existing disk without a spare, since when a mirror is started all the data from one partition is copied to the other. How to accomplish this is explained here. I still suggest having a spare disk, but I'm pretty conservative that way.
First we need to partition the disks that will be in the mirror:
Now we need to create the mirror sets and create the filesystems:
Setting Up Booting of the RAID Volume
With all the data migrated to raid sets, you now need to enable a few things for your system to successfully boot from the raid set. Two things are needed, your initrd must preload the md driver and LILO must know that the boot disk is a raid volume (this allows booting with a failed disk).
To create an initrd image with the md driver preloaded run the command: mkinitrd --preload=md --preload=loop --preload=xfs --preload=XXX imagename. What devices you need to preload needs to be determined for each system. For starters you must preload any modules that are needed to access the boot disks. Then you must preload the filesystems. Note that the loop device is needed to remount your root filesystem read-write if it is initially mounted read-only at boot.
Now you need to edit /etc/lilo.conf and make a new section that is a copy of your current preferred boot section, with the initrd changed to the one you just made. Since I'm keeping my old boot disk just in case the sky falls, I edited lilo.conf on the new mirror pair mounted under /mnt/newroot. You also need to update the boot= line to have /dev/md0 and add raid-auto, which tells lilo to update both disks boot sectors. Now, to run lilo, you need a full /dev tree and /proc mounted. Since I use devfs this meant doing:
With my 18 disks, the Lilo version that I came with Mandrake 9.2 took a segmentation fault and dumped core. The solution was to download and compile version 22.6 of Lilo, which fixes a problem with greater than 16 disks.
Reboot and Go
Now we're all ready to reboot onto the mirror disks. Shutdown the system and pull the original boot disk and reboot. Lilo will start and you will see your new default mirror start. When the system boots log in a type /sbin/mount. All non-lvm volumes should be mounted on /dev/md?.
All volume groups should be using the md devices. Run /sbin/vgdisplay -v and verify that all volume groups are using /dev/md? instead of a physical disk.
A note on volume group naming
Volume groups can be named anything at all, however it is best to have a scheme that will allow you to identify your volume groups in your administration scripts. I use the name vgXX (where XX is a two digit number starting at 00). This allows for 100 volume groups, which is probably enough for most situations, using hexadecimal digits would up this to 256 volume groups.
As with volume groups, logical volumes can be named anything. Some people like to name their logical volumes after the mount points they are used for. /dev/vg00/var would be mounted at /var for instance. I prefer to keep the names the generic lvolX, as things inevitably change and sticking to generic names keeps things neat over time. Mount points should be documented separately in your configuration documentation.