search

cupcakearmy / autorestic

Config driven, easy backup cli for restic.
https://autorestic.vercel.app/
Apache License 2.0
1.16k stars 68 forks source link

Docker volume backup not working from within container #334

Closed YurNaybor closed 4 months ago

YurNaybor commented 8 months ago

Describe the bug I did not managed to get backup of Docker volumes working with cupcakearmy/autorestic in Docker.

Expected behavior Autorestic backing up Docker volumes.

Environment

Additional context

Autorestic log:

Using config:    /etc/autorestic/.autorestic.yml
Using env:   /etc/autorestic/.autorestic.env
Using lock:  /etc/autorestic/.autorestic.lock.yml

    Backing up location "dummy"    

Running hooks
> echo "Before"
> Executing: /bin/bash -c echo "Before"
Before

Backend: local
> Executing: docker volume inspect autorestic_dummy-location

Running hooks
> echo "After"
> Executing: /bin/bash -c echo "After"
After

Running hooks
> echo "Something went wrong"
> Executing: /bin/bash -c echo "Something went wrong"
Something went wrong

Autorestic configuration:

version: 2

global:
  all:
    cache-dir: /cache
  forget:
    keep-daily: 30
    keep-weekly: 52

locations:
  dummy:
    from: autorestic_dummy-location
    type: volume
    to:
      - local
    cron: '* * * * *'  # Every minute
    forget: prune
    hooks:
      before:
        - echo "Before"
      after:
        - echo "After"
      failure:
        - echo "Something went wrong"
      success:
        - echo "Well done!"

backends:
  local:
    type: local
    path: /backends/local

Autorestic invocation (in container):

export DOCKER_HOST=tcp://docker-socket-proxy:2375
while true; do autorestic -v -c /etc/autorestic/.autorestic.yml --ci cron; sleep 10; done

Docker volume listing:

root@xxx:/# docker volume ls
DRIVER    VOLUME NAME
local     autorestic_cache
local     autorestic_dummy-location
local     autorestic_local-backend

If I read the code correctly, Autorestic uses the Docker CLI to interact with Docker. Now, the Docker CLI is not included in the image, thus this cannot work:

root@xxx:/# docker exec -ti autorestic bash
docker:/$ docker
bash: docker: command not found

It is also strange to see no error message related to this in the Autorestic output. It's only the invocation of the failure hook ("Something went wrong") showing that there is an issue.

YurNaybor commented 8 months ago

I've built a custom image based on cupcakearmy/autorestic:1.7.9 and only added docker-cli via apk. This was enough to come one step further. Autorestic now could find the volume and started a docker run command.

Unfortunately, this also failed without error message. At least it printed the command and I was able, by running it manually, to find out that apparently it did not run restic init before running restic backup. I'm not sure if that's an configuration issue on my side or Autorestic bug. Do I have to run restic init or autorestic check manually (initially) when using cron?

Running init manually solved it an now Autorestic seems to run trough fine.

Though I am not sure if there should be output after "Executing...forget"

Using config:    /etc/autorestic/.autorestic.yml
Using env:   /etc/autorestic/.autorestic.env
Using lock:  /etc/autorestic/.autorestic.lock.yml

    Backing up location "dummy"    

Running hooks
> echo "Before"
> Executing: /bin/bash -c echo "Before"
Before

Backend: local
> Executing: /usr/bin/docker volume inspect autorestic_dummy-location
[
    {
        "CreatedAt": "2023-10-29T01:19:43-04:00",
        "Driver": "local",
        "Labels": {
            "com.docker.stack.namespace": "autorestic"
        },
        "Mountpoint": "/var/lib/docker/165536.165536/volumes/autorestic_dummy-location/_data",
        "Name": "autorestic_dummy-location",
        "Options": {},
        "Scope": "local"
    }
]
> Executing: /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume autorestic_dummy-location:/data --hostname docker --volume /backends/local:/repo --env RESTIC_REPOSITORY=/repo --env RESTIC_PASSWORD=topsecret cupcakearmy/autorestic:1.7.9 -c restic backup --tag ar:cron --tag ar:location:dummy /data

Running hooks
> echo "After"
> Executing: /bin/bash -c echo "After"
After

Running hooks
> echo "Something went wrong"
> Executing: /bin/bash -c echo "Something went wrong"
Something went wrong
root@xxx:/# /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume autorestic_dummy-location:/data --hostname docker --volume /backends/local:/repo --env RESTIC_REPOSITORY=/repo --env RESTIC_PASSWORD=topsecret cupcakearmy/autorestic:1.7.9 -c "restic backup --tag ar:cron --tag ar:location:dummy /data"
Fatal: unable to open config file: stat /repo/config: no such file or directory
Is there a repository at the following location?
/repo
root@xxx:/# /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume autorestic_dummy-location:/data --hostname docker --volume /backends/local:/repo --env RESTIC_REPOSITORY=/repo --env RESTIC_PASSWORD=topsecret cupcakearmy/autorestic:1.7.9 -c "restic init"
created restic repository b83abed212 at /repo

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
root@xxx:/# /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume autorestic_dummy-location:/data --hostname docker --volume /backends/local:/repo --env RESTIC_REPOSITORY=/repo --env RESTIC_PASSWORD=topsecret cupcakearmy/autorestic:1.7.9 -c "restic backup --tag ar:cron --tag ar:location:dummy /data"
no parent snapshot found, will read all files

Files:           2 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repository: 846 B (564 B stored)

processed 2 files, 0 B in 0:00
snapshot eb58e720 saved
Using config:    /etc/autorestic/.autorestic.yml
Using env:   /etc/autorestic/.autorestic.env
Using lock:  /etc/autorestic/.autorestic.lock.yml

    Backing up location "dummy"    

Running hooks
> echo "Before"
> Executing: /bin/bash -c echo "Before"
Before

Backend: local
> Executing: /usr/bin/docker volume inspect autorestic_dummy-location
[
    {
        "CreatedAt": "2023-10-29T01:19:43-04:00",
        "Driver": "local",
        "Labels": {
            "com.docker.stack.namespace": "autorestic"
        },
        "Mountpoint": "/var/lib/docker/165536.165536/volumes/autorestic_dummy-location/_data",
        "Name": "autorestic_dummy-location",
        "Options": {},
        "Scope": "local"
    }
]
> Executing: /usr/bin/docker run --rm --entrypoint ash --workdir /data --volume autorestic_dummy-location:/data --hostname docker --volume /backends/local:/repo --env RESTIC_PASSWORD=topsecret --env RESTIC_REPOSITORY=/repo cupcakearmy/autorestic:1.7.9 -c restic backup --tag ar:cron --tag ar:location:dummy /data

Running hooks
> echo "After"
> Executing: /bin/bash -c echo "After"
After

Running hooks
> echo "Well done!"
> Executing: /bin/bash -c echo "Well done!"
Well done!

  Forgetting for location "dummy"  

For backend "local"
> Executing: /usr/bin/restic --cache-dir /cache forget --tag ar:location:dummy --prune --keep-daily 30 --keep-weekly 52