A few weeks ago I've notices that my backups for Proxmox are constantly failing. Interestingly just for a specific VM.
While the backup is uploading my OPNsense is dropping its WAN-Connection and thus killing my backup upload. OPNsense is running on a different node and is not backup at the same time. Noting interesting is shown inside the system.log. Just on the screen. A screenshot is attached below. It sows the message
nd6_dad_timer: called with non-tentative address fe80:9::28c:6cff:feßa: 1319 (pppoe)
I have absolutely no idea what this is related to. DuckDuckGo didn't gave me any useful information. Also no-one has commented on my previous reddit posts at r/OPNsense and r/OPNsenseFirewall.
To Reproduce
I have absolutely no idea how to reproduce this.
Expected behavior
Not dropping the WAN-Connection.
Describe alternatives you considered
None. I don't even know what this error Is trying to tell me.
Screenshots
Console:
Relevant log files
Proxmox log:
111: 2022-06-07 18:10:35 INFO: Starting Backup of VM 111 (qemu)
111: 2022-06-07 18:10:35 INFO: status = running
111: 2022-06-07 18:10:35 INFO: VM Name: HomeAssisstant
111: 2022-06-07 18:10:35 INFO: include disk 'sata0' 'SSD:vm-111-disk-1' 32G
111: 2022-06-07 18:10:35 INFO: include disk 'efidisk0' 'SSD:vm-111-disk-0' 4M
111: 2022-06-07 18:10:36 INFO: backup mode: snapshot
111: 2022-06-07 18:10:36 INFO: ionice priority: 7
111: 2022-06-07 18:10:36 INFO: snapshots found (not included into backup)
111: 2022-06-07 18:10:36 INFO: creating Proxmox Backup Server archive 'vm/111/2022-06-07T16:10:35Z'
111: 2022-06-07 18:10:36 INFO: issuing guest-agent 'fs-freeze' command
111: 2022-06-07 18:10:38 INFO: issuing guest-agent 'fs-thaw' command
111: 2022-06-07 18:10:39 INFO: started backup task '47e6b8e6-b1c0-4a13-b6e0-897fd7b42adf'
111: 2022-06-07 18:10:39 INFO: resuming VM again
111: 2022-06-07 18:10:39 INFO: efidisk0: dirty-bitmap status: OK (drive clean)
111: 2022-06-07 18:10:39 INFO: sata0: dirty-bitmap status: OK (3.2 GiB of 32.0 GiB dirty)
111: 2022-06-07 18:10:39 INFO: using fast incremental mode (dirty-bitmap), 3.2 GiB dirty of 32.0 GiB total
111: 2022-06-07 18:10:42 INFO: 5% (192.0 MiB of 3.2 GiB) in 3s, read: 64.0 MiB/s, write: 62.7 MiB/s
111: 2022-06-07 18:10:45 INFO: 7% (240.0 MiB of 3.2 GiB) in 6s, read: 16.0 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:10:50 INFO: 8% (264.0 MiB of 3.2 GiB) in 11s, read: 4.8 MiB/s, write: 4.8 MiB/s
111: 2022-06-07 18:10:53 INFO: 9% (296.0 MiB of 3.2 GiB) in 14s, read: 10.7 MiB/s, write: 10.7 MiB/s
111: 2022-06-07 18:10:58 INFO: 10% (332.0 MiB of 3.2 GiB) in 19s, read: 7.2 MiB/s, write: 7.2 MiB/s
111: 2022-06-07 18:11:01 INFO: 11% (384.0 MiB of 3.2 GiB) in 22s, read: 17.3 MiB/s, write: 17.3 MiB/s
111: 2022-06-07 18:11:04 INFO: 12% (408.0 MiB of 3.2 GiB) in 25s, read: 8.0 MiB/s, write: 8.0 MiB/s
111: 2022-06-07 18:11:07 INFO: 14% (468.0 MiB of 3.2 GiB) in 28s, read: 20.0 MiB/s, write: 20.0 MiB/s
111: 2022-06-07 18:11:10 INFO: 15% (512.0 MiB of 3.2 GiB) in 31s, read: 14.7 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:11:13 INFO: 17% (572.0 MiB of 3.2 GiB) in 34s, read: 20.0 MiB/s, write: 20.0 MiB/s
111: 2022-06-07 18:11:16 INFO: 19% (620.0 MiB of 3.2 GiB) in 37s, read: 16.0 MiB/s, write: 16.0 MiB/s
111: 2022-06-07 18:11:19 INFO: 20% (660.0 MiB of 3.2 GiB) in 40s, read: 13.3 MiB/s, write: 13.3 MiB/s
111: 2022-06-07 18:11:22 INFO: 21% (704.0 MiB of 3.2 GiB) in 43s, read: 14.7 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:11:25 INFO: 22% (740.0 MiB of 3.2 GiB) in 46s, read: 12.0 MiB/s, write: 12.0 MiB/s
111: 2022-06-07 18:11:28 INFO: 24% (784.0 MiB of 3.2 GiB) in 49s, read: 14.7 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:11:31 INFO: 26% (852.0 MiB of 3.2 GiB) in 52s, read: 22.7 MiB/s, write: 22.7 MiB/s
111: 2022-06-07 18:11:35 INFO: 27% (900.0 MiB of 3.2 GiB) in 56s, read: 12.0 MiB/s, write: 12.0 MiB/s
111: 2022-06-07 18:11:38 INFO: 29% (960.0 MiB of 3.2 GiB) in 59s, read: 20.0 MiB/s, write: 20.0 MiB/s
111: 2022-06-07 18:11:41 INFO: 30% (988.0 MiB of 3.2 GiB) in 1m 2s, read: 9.3 MiB/s, write: 9.3 MiB/s
111: 2022-06-07 18:11:44 INFO: 31% (1016.0 MiB of 3.2 GiB) in 1m 5s, read: 9.3 MiB/s, write: 9.3 MiB/s
111: 2022-06-07 18:11:47 INFO: 32% (1.0 GiB of 3.2 GiB) in 1m 8s, read: 10.7 MiB/s, write: 10.7 MiB/s
111: 2022-06-07 18:11:50 INFO: 33% (1.1 GiB of 3.2 GiB) in 1m 11s, read: 10.7 MiB/s, write: 10.7 MiB/s
111: 2022-06-07 18:11:53 INFO: 34% (1.1 GiB of 3.2 GiB) in 1m 14s, read: 14.7 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:11:56 INFO: 37% (1.2 GiB of 3.2 GiB) in 1m 17s, read: 33.3 MiB/s, write: 33.3 MiB/s
111: 2022-06-07 18:11:59 INFO: 38% (1.2 GiB of 3.2 GiB) in 1m 20s, read: 12.0 MiB/s, write: 12.0 MiB/s
111: 2022-06-07 18:12:02 INFO: 39% (1.3 GiB of 3.2 GiB) in 1m 23s, read: 8.0 MiB/s, write: 8.0 MiB/s
111: 2022-06-07 18:12:05 INFO: 41% (1.3 GiB of 3.2 GiB) in 1m 26s, read: 21.3 MiB/s, write: 21.3 MiB/s
111: 2022-06-07 18:12:08 INFO: 42% (1.4 GiB of 3.2 GiB) in 1m 29s, read: 14.7 MiB/s, write: 13.3 MiB/s
111: 2022-06-07 18:12:11 INFO: 44% (1.4 GiB of 3.2 GiB) in 1m 32s, read: 18.7 MiB/s, write: 18.7 MiB/s
111: 2022-06-07 18:12:14 INFO: 46% (1.5 GiB of 3.2 GiB) in 1m 35s, read: 26.7 MiB/s, write: 26.7 MiB/s
111: 2022-06-07 18:12:17 INFO: 48% (1.5 GiB of 3.2 GiB) in 1m 38s, read: 18.7 MiB/s, write: 18.7 MiB/s
111: 2022-06-07 18:12:20 INFO: 49% (1.6 GiB of 3.2 GiB) in 1m 41s, read: 14.7 MiB/s, write: 14.7 MiB/s
111: 2022-06-07 18:12:23 INFO: 52% (1.7 GiB of 3.2 GiB) in 1m 44s, read: 25.3 MiB/s, write: 25.3 MiB/s
111: 2022-06-07 18:12:34 INFO: 52% (1.7 GiB of 3.2 GiB) in 1m 55s, read: 1.1 MiB/s, write: 1.1 MiB/s
111: 2022-06-07 18:12:34 ERROR: backup write data failed: command error: protocol canceled
111: 2022-06-07 18:12:34 INFO: aborting backup job
111: 2022-06-07 18:12:34 INFO: resuming VM again
111: 2022-06-07 18:12:34 ERROR: Backup of VM 111 failed - backup write data failed: command error: protocol canceled
Describe the bug
A few weeks ago I've notices that my backups for Proxmox are constantly failing. Interestingly just for a specific VM.
While the backup is uploading my OPNsense is dropping its WAN-Connection and thus killing my backup upload. OPNsense is running on a different node and is not backup at the same time. Noting interesting is shown inside the system.log. Just on the screen. A screenshot is attached below. It sows the message
I have absolutely no idea what this is related to. DuckDuckGo didn't gave me any useful information. Also no-one has commented on my previous reddit posts at r/OPNsense and r/OPNsenseFirewall.
To Reproduce
I have absolutely no idea how to reproduce this.
Expected behavior
Not dropping the WAN-Connection.
Describe alternatives you considered
None. I don't even know what this error Is trying to tell me.
Screenshots
Console:
Relevant log files
Proxmox log:
OPNsense log:
Additional context
Potatoes are amazing
Environment
OPNsense 22.1.8_1 (amd64, OpenSSL). KVM Guest via Proxmox (On a separate node. My backups are failing on a different one.)