digint / btrbk

Tool for creating snapshots and remote backups of btrfs subvolumes
https://digint.ch/btrbk/
GNU General Public License v3.0
1.67k stars 122 forks source link

btrbk still fails on btrfs receive errors even with compat_remote 'ignore_receive_errors' enabled #523

Closed cryobry closed 1 year ago

cryobry commented 1 year ago

I am trying to backup to an OpenWRT device without selinux but btrbk fails due to missing lsetxattr even with the compatibility option 'ignore_receive_errors' enabled.

ERROR: Failed to send/receive subvolume: /home/.snapshots/bryan.20221224T6969 -> router.lan:/mnt/backup/workstation/home/bryan.20221224T0969
ERROR: ... Command execution failed (exitcode=1)
ERROR: ... sh: btrfs send '/home/.snapshots/bryan.20221224T6909" | mbuffer -v 1 -q -m 512m | ssh —i '/home/bryan/.config/btrbk/id_ed25519' root@router.lan 'btrfs receive --max-errors=0 '\''/mnt/backup/workstation/home/"\"""
ERROR: ... lsetxattr .local/share/containers/storage/overlay/517d3e85b613196d4529f186993b02d90c68ceef1af304d99c2a282ccd9n1en,/di FF /var/Lib/ tpm2-tss/system/keystore system.posix_acl_default= failed: Not supported
ERROR: ... lsetxattr .local/share/containers/storage/overlay/517d3e85b61319f6d4529186993b02d90c68ceef1af304d99c2a282ccd981e0/di FF/run/tpm2-tss/eventlog system.posix_acl _default= failed: Not supported
[delete] target: router. lan: /mnt/backup/workstation/home/bryan.20221224T0969
WARNING: Deleted partially received (garbled) subvolume: router.lan:/mnt/backup/workstation/home/bryan.20221224T6909
ERROR: Error while resuming backups, aborting

I understand that this is largely an upstream issue, I'm just curious why --max-errors=0 isn't skipping the lsetxattr steps in the stream, or if there are any other things I can try to get send-receive working between these two devices.

cryobry commented 1 year ago

I'm going to test this again overnight using send_protocol '2'.

cryobry commented 1 year ago

When using send protocol '2', the btrfs send stream stops transferring shortly after starting without any errors.

Creating backup: router.lan:/mnt/backup/workstation/home/bryan.20230104T2216
[send/receive] target: router. lan: /mnt/backup/workstation/home/bryan.20230104T2216
[send/receive] source: /home/.snapshots/bryan.20230104T2216

in@ 0.0 kiB/s, out @ 0.0 kiB/s, 744 MiB total, buffer 100% full]

Source machine:

Linux workstation 6.0.15-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Dec 21 18:33:23 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux btrfs-progs v6.1.2

Receive machine: Linux router 5.10.146 #0 SMP PREEMPT Fri Oct 14 22:44:41 2022 aarch64 GNU/Linux btrfs-progs v6.0.1

digint commented 1 year ago

The error states system.posix_acl_default= failed: Not supported, which most probably means that you don't have ACL support for btrfs in the kernel on the receiving host, but your sending host makes use of ACL's in the subvolume transferred. Check if your kernel has BTRFS_FS_POSIX_ACL enabled.

If you can't change the kernel on the target, you can disable ACL's by setting noacl in /etc/fstab. (Note that your distro might depend on this for some security features).

When using send protocol '2', the btrfs send stream stops transferring shortly after starting without any errors.

I believe you need kernel >= 6.0 on the receiving host as well for send protocol 2 to work.

I understand that this is largely an upstream issue, I'm just curious why --max-errors=0 isn't skipping the lsetxattr steps in the stream, or if there are any other things I can try to get send-receive working between these two devices.

No idea why --max-errors=0 has no effect here, maybe it's a bug in btrfs-progs?

Linux workstation 6.0.15-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Dec 21 18:33:23 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

LOL, github effectively links to issue #1 with your kernel version :D

cryobry commented 1 year ago

Indeed, this was caused by missing ACL support on the OpenWRT target. I recompiled OpenWRT with the CONFIG_KERNEL_BTRFS_FS_POSIX_ACL=y config option and all is working again.