Ignore parent dataset with --recursive.

[biohubgriznog]

Is there a way to configure a parent such that all children are replicated with --recursive, but the parent dataset is skipped?

Example use case:

Several servers have filesystems set up like

server1:
  pool/exports/instruments/NAME1
  pool/exports/instruments/NAME2

server2: 
  pool/exports/instruments/NAME3
  pool/exports/instruments/NAME4

...

A single server is used to provide backup space/target for these

backup-server:
  pool/backups/instruments/NAME1
  pool/backups/instruments/NAME2
  pool/backups/instruments/NAME3
  pool/backups/instruments/NAME4

I can set up a recursive rule on all the serverN systems to send pool/exports/instruments recursively to backup-server:pool/backups/instruments and that works fine except that they all throw an error that pool/exports/instruments have no matching snapshots. I'd like to not have that error show up but also not have to configure znapzend for every individual instrument share on every server.

We'd also like to avoid the workaround of using

backup-server:poool/backups/instruments/serverN/...

as leaving out that server information allows the backup hierarchy to nicely map to the namespace we build with autofs for users to see.

[oetiker]

Znapzend is using native zfs send and receive. The receive function does not allow to 'merge' two streams. If something like what you are looking for should be implemented, znapzend would have todo the recursive sending manually.

Patches to implement something like this in a 'nice' way are certainly welcome!

[biohubgriznog]

Sorry, I didn't explain it well. I don't want to merge the child streams, just have the parent specify the znapzend config so the children inherit it with an option to ignore the parent in the actual sync. The way syncoid does with --skip-parent.

[biohubgriznog]

I'm still not happy with my explanation, so from a real world example, I have this on serverA:

# zfs list -r -o name datapool/exports/instruments
NAME
datapool/exports/instruments
datapool/exports/instruments/MERSCOPE-01
datapool/exports/instruments/MiSeq-01
datapool/exports/instruments/MinION
datapool/exports/instruments/NextSeq-01
datapool/exports/instruments/NextSeq-02
datapool/exports/instruments/NextSeq-03
datapool/exports/instruments/NextSeq-04
datapool/exports/instruments/NovaSeq-01
datapool/exports/instruments/NovaSeq-02
datapool/exports/instruments/Sequel_IIe
datapool/exports/instruments/iSeq100-01

I have a znapzend config on datapool/exports/instruments :

dst_01-backupserver=root@backupserver:datapool/exports/backups/hpc/instruments
dst_01-backupserver_plan=1weeks=>1days,1months=>1weeks,1years=>1months
enabled=off
mbuffer=off
mbuffer_size=1G
post_znap_cmd=off
pre_znap_cmd=off
recursive=on
src=datapool/exports/instruments
src_plan=1days=>4hours,1weeks=>1days,1months=>1weeks,1years=>1months
tsformat=%Y-%m-%dT%H:%M:%SZ
zend_delay=0

Now when znapzend runs for this dataset I get an error because backupserver:datapool/exports/backups/hpc/instruments did not originate from serverA so there are no common snapshots, but all the children send/recv without any issues as expected.

What I want to do, in config terms, is have an option

skip_parent=on

which will tell znapzend to just get the config from the parent but not try to actually send/recv for it.

[oetiker]

AFAIK znapzend uses the recursive send function of zfs send ... but maybe one of the patches we integrated of late adds other functionality which would allow for what you are looking for, so by all means, hack away!

[biohubgriznog]

I'm not seeing a recursive option in zfs send maybe that's not ported to Linux? And since the children still send/recv after the parent fails with an error, I have been assuming znapzend handles each as a separate process.

Unfortunately I don't speak Perl, so even if I did hack together PR I'm betting it wouldn't be very pretty.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.