search

tasket / wyng-backup

Fast backups for logical volumes & disk images
GNU General Public License v3.0
251 stars 16 forks source link

Very cool project. Support for ZFS volumes? #110

Open Rudd-O opened 2 years ago

Rudd-O commented 2 years ago

This is a really cool project. I wonder how viable it would be to suck up ZFS send / receive streams (or large files) to back them up using Wyng? It should be just as fast or faster than looking at LVM snapshot metadata.

tasket commented 2 years ago

Thanks! Although I setup an Ubuntu server on ZFS I still know little about managing the fs. My inclination is to first go for Btrfs source volume support because of my experience with it. Note that Qubes doesn't appear to support ZFS natively (or that is the impression I got).

Guessing at the relative speed on thin-provision LVM (which is like a simplified fs on top of LVM) I wouldn't bet on any one horse, although thin LVM might have an edge being one of the simplest 'datacow' systems available.

Rudd-O commented 2 years ago

ZFS is much simpler than LVM (for one, there are no layers, much like btrfs is). In fact,if you can imagine btrfs but for block devices, that's what ZFS volumes are. ZFS is capable of instantaneously giving you the diff between two snapshots of a volume (or file system), as well as giving you a bit-exact replication stream between two snapshots as well (sort of like a git diff between two commits, but for binary data). That should be ingestable by Wyng.

tasket commented 1 year ago

You may wish to check out the Btrfs+XFS support that has been added to the Wyng 04wip branch. Since local storage functions were abstracted to make this possible, I suspect adding ZFS wouldn't be too much effort. What is needed is a way to obtain extent lists or similar metadata for disk images so that different snapshots can be compared.

tlaurion commented 7 months ago

From https://github.com/tasket/wyng-backup/issues/110#issuecomment-1477196667

I suspect adding ZFS wouldn't be too much effort. What is needed is a way to obtain extent lists or similar metadata for disk images so that different snapshots can be compared.

@Rudd-O any hints for zfs?