search

bellhyve / zelta

ZFS tools used for data migration and backup management.
Other
38 stars 6 forks source link

Add `zfs list` cache and prediction option to provide more varied replication options #2

Open bellhyve opened 10 months ago

bellhyve commented 10 months ago

There are two problems here:

Since the whole point of this thing is to be fast, we should consider this OpenZFS PR and stick to name, guid, and createtxg to prevent going into loop hell. This means our "written" estimate should only be enabled with -v or something.

bellhyve commented 10 months ago

That's fixed, but we should probably add caching. In zelta-match.awk, the"source" and "target" variables have been changed to hashes suitable for cache filenames from [user@][host:][volume] endpoint strings that are used in all utilities.

bellhyve commented 10 months ago

I switch the hashing to just dumb gsub of non-alphanumeric because I was getting collisions with the hash, especially for short dataset names. I still might consider caching.

bellhyve commented 10 months ago

Added zelta match -w to compute write sizes and as a utility function for zelta sync -s[s] which only snaps when written. That might be useful here. (One -s doesn't snapshot when written but continues to replicate newer snapshots. -ss will only snapshot and replicate if the volume has been written to.

bellhyve commented 9 months ago

Not just "zfs list" but also "zpool upgrade -v" options. That could be useful because we can auto-add or drop -w for encrypted sets while automatically maintaining compatibility with older sets that don't support -w...