search

oetiker / znapzend

zfs backup with remote capabilities and mbuffer integration.
www.znapzend.org
GNU General Public License v3.0
607 stars 137 forks source link

Raw (encrypted) send failure #491

Closed zdykstra closed 3 years ago

zdykstra commented 4 years ago

When attempting to use znapzend to send a snapshot on a pool that's fully encrypted to a remote pool that is unencrypted, I'm receiving an error, even though the snapshot(s) do arrive on the destination without issue.

Sender details: Void Linux, ZFS 0.8.4

# zfs get encryptionroot,encryption,keyformat,keylocation,keystatus zroot
NAME   PROPERTY        VALUE                      SOURCE
zroot  encryptionroot  zroot                      -
zroot  encryption      aes-256-gcm                -
zroot  keyformat       passphrase                 -
zroot  keylocation     file:///etc/zfs/zroot.key  local
zroot  keystatus       available                  -

# zfs list
NAME                         USED  AVAIL     REFER  MOUNTPOINT
zroot                       28.4G   433G      192K  none
zroot/ROOT                  16.0G   433G      192K  none
zroot/ROOT/void.2020.06.22  8.24G   433G     8.24G  /
zroot/home                  12.4G   433G     12.4G  /home

# zfs list -t snapshot
no datasets available

# znapzendzetup list
*** backup plan: zroot/ROOT ***
           dst_0 = root@rethe:zroot/backups/auri
      dst_0_plan = 60days=>4hours
         enabled = on
         mbuffer = off
    mbuffer_size = 1G
   post_znap_cmd = off
    pre_znap_cmd = off
       recursive = on
             src = zroot/ROOT
        src_plan = 30days=>4hours
        tsformat = %Y-%m-%d-%H%M%S
      zend_delay = 0

Receiver details: Void Linux, ZFS 0.8.4. For the purposes of this test, zroot/backups has been destroyed, so we're starting from scratch.

Full log of a run

# znapzend --loglevel=debug --features=sendRaw --autoCreation --runonce=zroot/ROOT
[2020-06-29 21:11:22.78305] [23015] [info] znapzend (PID=23015) starting up ...
[2020-06-29 21:11:22.78330] [23015] [info] refreshing backup plans for dataset "zroot/ROOT" ...
cannot open 'zroot/backups/auri': dataset does not exist
cannot open 'zroot/backups/auri': dataset does not exist
[2020-06-29 21:11:24.62637] [23015] [info] creating destination dataset 'root@rethe:zroot/backups/auri'...
[2020-06-29 21:11:24.62740] [23015] [info] found a valid backup plan for zroot/ROOT...
[2020-06-29 21:11:24.63057] [23015] [info] znapzend (PID=23015) initialized -- resuming normal operations.
[2020-06-29 21:11:24.63409] [23015] [debug] snapshot worker for zroot/ROOT spawned (23046)
[2020-06-29 21:11:24.63684] [23046] [info] creating recursive snapshot on zroot/ROOT
[2020-06-29 21:11:24.69364] [23046] [info] checking for explicitly excluded ZFS dependent datasets under 'zroot/ROOT'
[2020-06-29 21:11:24.74121] [23015] [debug] snapshot worker for zroot/ROOT done (23046)
[2020-06-29 21:11:24.74296] [23015] [debug] send/receive worker for zroot/ROOT spawned (23084)
[2020-06-29 21:11:24.74378] [23084] [info] starting work on backupSet zroot/ROOT
[2020-06-29 21:11:24.75973] [23084] [debug] sending snapshots from zroot/ROOT to root@rethe:zroot/backups/auri
cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one
[2020-06-29 21:11:25.11679] [23084] [warn] ERROR: cannot send snapshots to zroot/backups/auri on root@rethe
[2020-06-29 21:11:25.11771] [23084] [debug] sending snapshots from zroot/ROOT/void.2020.06.22 to root@rethe:zroot/backups/auri/void.2020.06.22
cannot open 'zroot/backups/auri/void.2020.06.22': dataset does not exist
[2020-06-29 21:14:16.76205] [23084] [warn] ERROR: suspending cleanup source dataset because 1 send task(s) failed:
[2020-06-29 21:14:16.76263] [23084] [warn]  +-->   ERROR: cannot send snapshots to zroot/backups/auri on root@rethe
[2020-06-29 21:14:16.76283] [23084] [info] done with backupset zroot/ROOT in 172 seconds
[2020-06-29 21:14:16.76750] [23015] [debug] send/receive worker for zroot/ROOT done (23084)

Filesystems on remote host

# zfs list | grep backups
zroot/backups                                                                       8.24G  1.64T       96K  none
zroot/backups/auri                                                                  8.24G  1.64T       96K  none
zroot/backups/auri/void.2020.06.22                                                  8.24G  1.64T     8.24G  none
# zfs list -t snapshot | grep backups
zroot/backups/auri/void.2020.06.22@2020-06-29-211124                                             0B      -     8.24G  -

Is there a step I'm missing or a configuration option I'm otherwise not enabling? Any insight into this would be greatly appreciated.

Edit: As an aside, it's great to see Perl and Mojolicious in use in the wild!

crabique commented 4 years ago

See https://github.com/oetiker/znapzend/blob/master/lib/ZnapZend/ZFS.pm#L355

     -F      Force a rollback of the file system to the most recent snap-
             shot before performing the receive operation. If receiving an
             incremental replication stream (for example, one generated by
             "zfs send -R -Fi -iI"), destroy snapshots and file systems
             that do not exist on the sending side.

The -F argument to zfs recv is always used by znapzend, so it is not possible to back up encrypted datasets into non-encrypted destination, as can be seen from your log:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one
zdykstra commented 4 years ago

See https://github.com/oetiker/znapzend/blob/master/lib/ZnapZend/ZFS.pm#L355

     -F      Force a rollback of the file system to the most recent snap-
             shot before performing the receive operation. If receiving an
             incremental replication stream (for example, one generated by
             "zfs send -R -Fi -iI"), destroy snapshots and file systems
             that do not exist on the sending side.

The -F argument to zfs recv is always used by znapzend, so it is not possible to back up encrypted datasets into non-encrypted destination, as can be seen from your log:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one

Yes, that's the crux of this problem - which is why opened the issue. If I manually create an encrypted filesystem on the remote side zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase zroot/backups/auri I still get the same error, even though I'm sending an encrypted snapshot to an encrypted filesystem.

Is there any way around this? If not, I don't see how I can use znapzend to send snapshots of my laptop to a remote host and still have them be encrypted at rest.

hansoostendorp commented 4 years ago

I think this is related to the issue I just submitted: #495 . When the destination dataset doesn't exist and you use autoCreation, ZnapZend will create a non encrypted destination dataset. But you can't send encrypted snapshots to a non encrypted dataset. The way around this, is to send the first encrypted snapshot yourself without the -F flag to a destination dataset that doesn't exist yet. After this, ZendZnap can send following snapshots without problems.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.