tasket / wyng-backup

Fast backups for logical volumes & disk images
GNU General Public License v3.0
249 stars 16 forks source link

Restore lvm backups on brtfs? #166

Closed tlaurion closed 6 months ago

tlaurion commented 1 year ago

Asking this wishing for a "yes, possible" in the goal of restoring 4.1 templates and vms backuped from 4.1 in lvm mode into 4.2 into brtfs. Seems like the issues related to perf won't be fixed soon (4k templates etc) where BTRFS just gets a perf boost on 4.2 by default.

Any insights on how/if that is possible @tasket ?

tasket commented 1 year ago

@tlaurion Its certainly possible now with extra steps taken by the user.

The only issue I'm aware of is (in the context of a system like Qubes) the volume names will differ almost completely. OTOH, for non-Qubes systems this may not pose any problem at all and volumes might be sent and received without changing volume names, depending on the admin's preferred storage arrangements.

Let's say you have a Qubes reflink storage pool at '/mnt/btrfs/libqubes' and the dirs 'appvms' and 'vm-templates' are directly under that. In this case you almost certainly want to make '/mnt/btrfs/libqubes' a Btrfs subvolume and use that path for wyng --local. Volumes names, therefore, will be relative to that local path much the same way they are relative to an LVM volume group. The only difference is that subdirs are inevitable, and Wyng must accommodate them in the volume names.

Hence...

# wyng list

Volumes:
 appvms/banking/private.img
 appvms/dev/private.img
 appvms/personal/private.img
 appvms/root-backup/private.img
 appvms/untrusted/private.img
 vm-templates/d11-kde/private.img
 vm-templates/d11-kde/root.img

If we want to restore banking from this archive into an LVM thin pool, we can either create the thin volume manually and then use wyng --save-to receive, or do a wyng rename appvms/banking/private.img vm-banking-private before running wyng receive. You can't receive the Btrfs volume directly to LVM if its name contains symbols like '/' because LVM doesn't support paths. (Also, there may be unicode chars in the volume name, which LVM also can't handle.)

Going in the opposite direction LVM to Btrfs: You can also rename before receiving, or you have the option of receiving directly using the LVM name and then doing mv /mnt/btrfs/libqubes/vm-banking-private /mnt/btrfs/libqubes/appvms/banking/private.img.

There is also a minor issue with permissions; Wyng currently can only preserve owner read/write status when going between LVM and reflink, but it will do this automatically.


The short answer is: Yes. It boils down to the difference in volume names. And I've already done that type of restoration between different pool types with Wyng; One must simply know the paths used by the different Qubes pool types and rename accordingly.


As for questions about making this work nicely under the VM concept directly, this will have to be addressed with wyng-util-qubes and the like. Wyng itself only recognizes VMs as communication conduits.

tasket commented 1 year ago

BTW, I think Btrfs has improved a lot but it still might be sub-optimal compared with XFS, which has less overhead than Btrfs but still supports reflinks. My understanding is there may still be a big difference in write amplification between the two.

tlaurion commented 1 year ago

@tasket is there advancement I'm not aware of?

tasket commented 1 year ago

@tlaurion Btrfs has a series of very substantial speed improvements in the kernel versions 6.1, 6.2, and 6.3. These have been available in Qubes 4.1 via the 'kernel-latest' package. I think Qubes 4.2 is starting out with kernel 6.1 or 6.2 IIRC.

tasket commented 6 months ago

Implemented in wyng-util-qubes.