jimsalterjrs / sanoid

These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
http://www.openoid.net/products/
GNU General Public License v3.0
3.14k stars 308 forks source link

syncoid: add -X send option in special options #934

Open ossimoi opened 5 months ago

ossimoi commented 5 months ago

-X is listed in allowed options but missing from parsespecialoptions so arguments never get parsed. This allows using ZFS native recursive send with excluded datasets.

Been running this for a while with --sendoptions="Rws X rpool/gentoo/var" without problems.

         -X, --exclude dataset[,dataset]…
             With -R, -X specifies a set of datasets (and, hence, their descendants), to be excluded from the send stream.  The root dataset may not be ex‐
             cluded.  -X a -X b is equivalent to -X a,b.
phreaker0 commented 2 months ago

thx

luispabon commented 2 months ago

Hmm I don't believe this is working for me:

~ syncoid --sendoptions="Rw X zroot/vms/storage-pool-ephemeral" --no-sync-snap "zroot/vms" "vault:backups/luis-fw16/zroot/vms"
NEWEST SNAPSHOT: autosnap_2024-09-17_08:29:47_daily
INFO: Sending oldest full snapshot zroot/vms@autosnap_2024-09-02_14:37:34_daily (~ UNKNOWN) to new target filesystem:
cannot send zroot/vms@autosnap_2024-09-02_14:37:34_daily recursively: snapshot zroot/vms/storage-pool-ephemeral@autosnap_2024-09-02_14:37:34_daily does not exist
warning: cannot send 'zroot/vms@autosnap_2024-09-02_14:37:34_daily': backup failed
0.00  B 0:00:00 [0.00  B/s] [<=>                                                                                     ]
cannot receive: failed to read from stream
CRITICAL ERROR:  zfs send -R -w  'zroot/vms'@'autosnap_2024-09-02_14:37:34_daily' | pv -p -t -e -r -b -s 0 | lzop  | mbuffer  -q -s 128k -m 16M | ssh      -S /tmp/syncoid-vault-1726569428-2984 vault ' mbuffer  -q -s 128k -m 16M | lzop -dfc | sudo zfs receive  -s -F '"'"'backups/luis-fw16/zroot/vms'"'"'' failed: 256 at /usr/sbin/syncoid line 549.

~ zfs list -r zroot/vms
NAME                               USED  AVAIL  REFER  MOUNTPOINT
zroot/vms                          180G   275G   168K  none
zroot/vms/etc                     3.22M   275G   756K  /etc/libvirt
zroot/vms/storage-pool             175G   275G   147G  /var/lib/libvirt
zroot/vms/storage-pool-ephemeral  4.83G   275G  4.83G  /var/lib/libvirt-ephemeral

~ grep "'X'" (which syncoid) -b5 -a5 
74223-      for my $char (split //, $value) {
74259-          if ($optionValue ne 0) {
74287-              return undef;
74305-          }
74310-
74311:          if ($char eq 'o' || $char eq 'x' || $char eq 'X') {
74366-              $lastOption = $char;
74391-              $optionValue = 1;
74413-          } else {
74425-              my %item = (
74442-                  "option"  => $char,

I manually patched the syncoid script. v2.2.0 on Ubuntu 24.04.

phreaker0 commented 2 months ago

@luispabon v2.2.0 doesn't have X whitelisted, you need to use the master version for now.

luispabon commented 2 months ago