Musings of a techie

When replacing (and upgrading) disks, pre-failure, I prefer backing up the original data first. Although the original disks are in a RAID0 (mirror) array, where available, I prefer to make a backup despite the fact I should be able to swap the old disk for a newer (and larger) disk without impact. This covers the case where the RAID0 array is able to read the data while both disks have failures in different locations, but removing a disk to upgrade would ultimately cause data-loss.

I have made use of LVM under Linux for decades. Segregating files onto separate logical-volumes (partitions) means that smaller filesystems can be repaired more quickly, while restricting the impact of data affected (or services down on a server), and the ability to extend logical-volumes has been a lifesaver. However, I haven’t made much use of LVM snapshots. LVM snapshots allow one to take an instant snapshot/copy (static, like a photo) of a logical-volume, and allow the original LV to continue being used/changed while one mounts and takes a backup of the snapshot. This is of most benefit when backing up volumes containing database files where consistency is necessary. Without such a snapshot (or stopping the database) individual DB files are likely to be changing as they are backed up, meaning a restored database will either be corrupt, or inconsistent.

In this particular case I want to make a copy of an LV (logical-volume) containing rsnapshot backups of multiple servers. rsnapshot runs every ~2 hours, and I know this backup will take a couple of days (2d4h30m) for 1.6TB of backups.

Rsnapshot gratuitously uses hard-links to reduce utilisation, so a copy needs to maintain hard-links. I would normally use rsync -H but, in this case, there is a large number of files/directories (19.5 million inodes) and rsync would uses a lot of RAM to complete the sync. This server has a small amount of RAM (2GB), and rsync crashed after 1-2 days with an out-of-memory error. The alternative is cp -a which doesn’t have the same RAM requirements.


root@stor01:~# lvcreate -L100G -s -n bkp_rsnapshot /dev/stor01_vg/rsnapshot_lv
Using default stripesize 64.00 KiB.
Logical volume "bkp_rsnapshot" created.
root@stor01:~# mount -r /dev/stor01_vg/bkp_rsnapshot /mnt/bkp

real 3142m7.637s
user 20m10.877s
sys 381m17.791s
Wed 8 Mar 01:41:51 GMT 2023
root@stor01:~# umount /mnt/bkp
root@stor01:~# lvremove /dev/stor01_vg/bkp_rsnapshot
Do you really want to remove and DISCARD active logical volume stor01_vg/bkp_rsnapshot? [y/n]: y
Logical volume "bkp_rsnapshot" successfully removed
root@stor01:~#


The source of an example is always useful, this from the LVM HOWTO on The Linux Documentation Project website.

Now that is done, I can be comfortable I have backups (albeit a couple of days old) to get me out of a hole if the disk replacement causes a volume failure.

From back when (we)blogs started I had considered writing one. That thought didn’t last too long when I saw what rubbish most people wrote, and considered what pearls of wisdom I had to offer the Internet community (in short, not much). I consider blogs to be like a diary – documentation about point-in-time thoughts or actions rather than being for procedures, reviews, etc.

From time to time I have reconsidered and come away with the same answer as above, even when my brother asked me to set up a blog on his website. I’ve generally preferred to document stuff on a wiki, as that seems a more logical way to organise and provide my information for people.

Sometimes, however, there have been times when I’ve considered the best software or hardware to use for something (eg what mobile ‘phone to get), or noted down a procedure thinking it was a one-off .. and then, months or more later I’ve wondered why I made that particular decision.

This is the reason I am starting this blog .. for me as a personal reference. If others gain use from it (aside from abusers) then fine.

Let’s see where this takes me …