Bug: Docker volume backup to B2 fails ("context deadline exceeded")

Describe the bug When trying to backup a local docker volume to Backblaze B2 it after ~60 seconds with the following error: Fatal: unable to open repository at b2:backup-bucket-test: context deadline exceeded

TL;DR: Only the combination of B2 as backend + docker volume as location is broken. Backing up a local folder location to B2 works fine. Backing up a docker volume to a local backend works fine.

Full console output:

> Executing: /usr/bin/docker volume inspect backup-source-container-test-volume
[
    {
        "CreatedAt": "2024-03-07T14:28:38Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "backup-source-container-test",
            "com.docker.compose.version": "2.20.2",
            "com.docker.compose.volume": "backup-source-container-test-volume"
        },
        "Mountpoint": "/var/lib/docker/volumes/backup-source-container-test-volume/_data",
        "Name": "backup-source-container-test-volume",
        "Options": null,
        "Scope": "local"
    }
]
> Executing: /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume backup-source-container-test-volume:/data --hostname my-server-1 --env B2_ACCOUNT_KEY=<redacted> --env B2_ACCOUNT_ID=<redacted> --env RESTIC_PASSWORD=<redacted> --env RESTIC_REPOSITORY=b2:backup-server1-test cupcakearmy/autorestic:1.7.11 -c restic backup --tag ar:location:docker-volume-test /data

Location definition:

locations:
  docker-volume-test:
    from:
    - backup-source-container-test-volume
    type: volume
    to:
    - b2-bucket-test-1-root-folder
    hooks:
      dir: ""
      before: []
      after: []
      success: []
      failure: []
    cron: ""
    options:
      forget:
        keep-daily:
        - 4
        keep-hourly:
        - 3
        keep-last:
        - 5
        keep-monthly:
        - 12
        keep-weekly:
        - 1
        keep-within:
        - 14d
        keep-yearly:
        - 7
    copyoption: {}

Backend definition:

backends:
  b2-bucket-test-1-root-folder:
    type: b2
    path: backup-server1-test
    key: <redacted>
    env:
      b2_account_id: <redacted>
      b2_account_key: <redacted>
    rest:
      user: ""
      password: ""
    options: {}

Further tests I did in order to rule out that my B2 is not working or the docker volume is the issue:

If I change the backend of my location docker-volume-test to a local backend, it backs up just fine
If I back up a local folder with my backend b2-bucket-test-1-root-folder, it also backs up fist fine

Expected behavior

Backing up a docker volume to B2 works just as it does with a local backend for example

Environment

OS:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Version:

$ autorestic --version
autorestic version 1.7.11

Additional context If anybody could run a test to backup a docker volume to Backblaze (or any other cloud provider), that'd be awesome to compare configurations. Any helps is much appreciated and thanks for your time reading this :)

i'm not sure if this is related, but i am seeing a weird, seemingly random behaviour of stuck backup jobs of container volumes. the container is (based on cupcakearmy/autorestic:1.7.7) is running but seemingly not doing anything. volume size is about 5mb, should take mere seconds. i can see it because neither failure nor success hock gets triggered.

i'm not seeing any error messages or obvious entries in the systemd journal

autorestic version 1.7.7

os Ubuntu 22.04.4 LTS

version: 2
locations:
  redacted_backups:
    hooks:
      before:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Starting backup for location: redacted" https://redacted/start'
      failure:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Backup failed for location: redacted" https://redacted/fail'
      success:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Backup successful for location: redacted" https://redacted'
    from: /srv/backup/redacted
    to:
      - b2
    cron: "0 3 * * *"
  redacted_backups:
    hooks:
      before:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Starting backup for location: redacted" https://redacted/start'
      failure:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Backup failed for location: redacted" https://redacted/fail'
      success:
        - 'curl -m 10 --retry 5 -X POST -H "Content-Type: text/plain" --data "Backup successful for location: redacted" https://redacted'
    from: redacted
    type: volume
    to:
      - b2
    cron: "0 4 * * *"

backends:
  b2:
    type: b2
    path: 'redacted'
    key: "redacted"
    env:
      B2_ACCOUNT_ID: "redacted"
      B2_ACCOUNT_KEY: "redacted"⏎

cupcakearmy / autorestic

Bug: Docker volume backup to B2 fails ("context deadline exceeded") #357