search

djmaze / resticker

Run automatic restic backups via a Docker container.
https://hub.docker.com/r/mazzolino/restic/
Apache License 2.0
479 stars 65 forks source link

Question about Resticker #145

Open killmasta93 opened 1 year ago

killmasta93 commented 1 year ago

HI I was wondering if someone could shed some light on the questions i have Currently i have working Resticker but there is a few things could not get it working

1) the mounting seems that its only be able to mount in the container, is it possible to mount the files in the host?

docker-compose exec sftp_backup restic mount /media/view

2) The restore part i was able to correctly run it but not sure if this is the correct method


docker-compose down
then i remove the HTML folder 
create the html folder
docker-compose up -d
docker-compose exec sftp_backup restic -r sftp:root@192.168.7.244:/home/new_backup restore d91884a2 --include /tempwp/html --target /

This lets me restore the files into the HTML folder of the host, but using the docker-compose up after created the HTML folder creates an empty backup i tried to delete the HTML while the restic container are turned on and run the restore but does not restore not sure if i missed something

3) i see the restic creates logs in each container in .json format is it possible to keep a certain about of logs?

4) I also configured correctly the SMTP alert but cant seem to get information of the log in the email attaching the docker-compose below 

version: "3.3"

services:
  sftp_backup:
    image: mazzolino/restic
    restart: always
    hostname: wpsftp-backup
    privileged: true
    environment:
      RUN_ON_STARTUP: "true"
      PRE_COMMANDS: |-
           sleep 10
           docker exec root_db_1 mysqldump -u root -pthepassword wordpress | restic backup --tag mysqlbackupsft --stdin --stdin-filename wordpress.sql
           docker exec compose_db_1 pg_dump -U postgres -d openproject | restic backup --tag pgsqlbackupsftp --stdin --stdin-filename openproject.sql
           restic unlock
      POST_COMMANDS_FAILURE: |-
        curl -X POST --data "{\"title\": \"Backup failed\", \"body\": \"\"}" http://notify:5000
      POST_COMMANDS_SUCCESS: |-
        curl -X POST --data "{\"title\": \"Backup successfully completed\", \"body\": \"\"}" http://notify:5000
      BACKUP_CRON: "0 30 3 * * *"
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
      RESTIC_BACKUP_SOURCES: /tempwp/html
      RESTIC_BACKUP_ARGS: >-
        --tag docker-volumessft
        --verbose
      RESTIC_FORGET_ARGS: >-
        --keep-last 70
      TZ: America/Bogota
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    volumes:
      - /mnt/:/mnt/
      - /tempwp/html/:/tempwp/html/
      - /root/.ssh/:/root/.ssh/
      - /var/run/docker.sock:/var/run/docker.sock
      - /lib/modules:/lib/modules
    networks:
      - notification

  sftp-prune:
    image: mazzolino/restic
    hostname: wpsftp-prune
    environment:
      RUN_ON_STARTUP: "true"
      PRUNE_CRON: "0 0 4 * * *"
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
    volumes:
      - /root/.ssh/:/root/.ssh/

  sftp_check:
    image: mazzolino/restic
    hostname: wpsftp-check
    environment:
      RUN_ON_STARTUP: "false"
      CHECK_CRON: "0 15 5 * * *"
      RESTIC_CHECK_ARGS: >-
        --read-data-subset=10%
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
      TZ: America/Bogota
    volumes:
      - /root/.ssh/:/root/.ssh/

  notify:
    image: mazzolino/apprise-microservice:0.1
    environment:
      NOTIFICATION_URLS: mailtos://sistemas4:mypassword@domain.com.co/sistemas4@domain.com/sistemas2@domain.com?smtp=mail.domain.com.co&port=465&name=Docker%20Backup%20Notification
    networks:
      - notification

networks:
  notification:

Thank you

djmaze commented 1 year ago
  1. It might be possible to achieve that if you mount a folder from the host into the container with a volume using a special bind-propagation, see https://docs.docker.com/storage/bind-mounts/. This sounds hairy and I am not proficient in this.
  2. Looks to me like this should work, but I guess I don't understand you exactly. The backup is always empty?
  3. Does it? That is new to me. Everything is logged to stdout/stderr so everything goes straight to the docker logs. So you can use docker-compose logs (or docker logs) to get them.
  4. It is currently not possible to send the log output using post commands. I guess it should be possible to implement this in the future though. As a workaround, you might be able to run docker logs sftp_backup in your post commands and use the output, something like this: curl -X POST --data "{\"title\": \"Backup successfully completed\", \"body\": \"$(docker logs sftp_backup)\"}" http://notify:5000. Not sure about this though.
killmasta93 commented 1 year ago

Thanks for the reply, 1) will check it out and post back 2) im a bit confused on the restore part what would be the correct procedure to restore? for example i tried creating a scenario deleting the HTML folder which the restic docker backs it up as soon as i delete the html
i then try to restore with this command

docker-compose exec sftp_backup restic -r sftp:root@192.168.7.244:/home/new_backup restore 167bcec1 --include /tempwp/html --target /

i then recheck the /tempwp and i don't see the html folder restored

i then try to create the html folder manually and re run the restore command but still does not show no information in the html folder

3) yeah i was checking the logs in the location /var/lib/docker/containers/fac437a62fbf7fe8289a970f7f4fac7badfc2810a7c08463494146d781aa13b0# cat fac437a62fbf7fe8289a970f7f4fac7badfc2810a7c08463494146d781aa13b0-json.log it seems that if the docker restarts or stays on it keeps the logs but if i run docker-compose down it removes that information is there a way i can make the logs persistent?

4) as for the output it seems that its giving an error

ERROR: yaml.parser.ParserError: while parsing a block mapping
  in "./docker-compose.yml", line 10, column 7
expected <block end>, but found '<block mapping start>'
  in "./docker-compose.yml", line 20, column 9

Thank you again

ifeiwu commented 1 year ago

I also have this problem

killmasta93 commented 1 year ago

I also have this problem

which problem do you have?

djmaze commented 1 year ago

@killmasta93

  1. The restore command you posted seems correct to me. If you have the /tempwp/html volume mounted as shown in your config then this should work. Not sure what is going wrong there.. You might exec into the container and look if the hosts' /tempwp/html folder is really correctly available in there.
  1. it seems that if the docker restarts or stays on it keeps the logs but if i run docker-compose down it removes that information is there a way i can make the logs persistent?

You have a variety of options for configuring logging differently

  1. as for the output it seems that its giving an error

Mhh.. Your config as you showed above seems correct. Did you change anything in your docker-compose.yml in the meantime?

killmasta93 commented 1 year ago

Hi @djmaze thank you so much for the reply, 2. allright these were the steps that i took which worked as for the example i deleted the html folder in the location of /tempwp/ it seems that i needed to turn off the wordpress container first , then restart the resticker container then run docker-compose exec sftp_backup restic -r sftp:root@192.168.7.244:/home/new_backup restore 167bcec1 --include /tempwp/html --target /

not sure if this is the correct way

  1. thank you will take alook at it

  2. as for the config same haven't changed it

version: "3.3"

services:
  sftp_backup:
    image: mazzolino/restic
    restart: always
    hostname: wpsftp-backup
    privileged: true
    environment:
      RUN_ON_STARTUP: "true"
      PRE_COMMANDS: |-
           sleep 10
           docker exec root_db_1 mysqldump -u root -pthepassword wordpress | restic backup --tag mysqlbackupsft --stdin --stdin-filename wordpress.sql
           docker exec compose_db_1 pg_dump -U postgres -d openproject | restic backup --tag pgsqlbackupsftp --stdin --stdin-filename openproject.sql
           restic unlock
      POST_COMMANDS_FAILURE: |-
        curl -X POST --data "{\"title\": \"Backup failed\", \"body\": \"\"}" http://notify:5000
      POST_COMMANDS_SUCCESS: |-
#        curl -X POST --data "{\"title\": \"Backup successfully completed\", \"body\": \"\"}" http://notify:5000
        curl -X POST --data "{\"title\": \"Backup successfully completed\", \"body\": \"$(docker logs sftp_backup)\"}" http://notify:5000
      BACKUP_CRON: "0 30 3 * * *"
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
      RESTIC_BACKUP_SOURCES: /tempwp/html
      RESTIC_BACKUP_ARGS: >-
        --tag docker-volumessft
        --verbose
      RESTIC_FORGET_ARGS: >-
        --keep-last 70
      TZ: America/Bogota
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    volumes:
      - /mnt/:/mnt/
      - /tempwp/html/:/tempwp/html/
      - /root/.ssh/:/root/.ssh/
      - /var/run/docker.sock:/var/run/docker.sock
      - /lib/modules:/lib/modules
    networks:
      - notification

  sftp-prune:
    image: mazzolino/restic
    hostname: wpsftp-prune
    environment:
      RUN_ON_STARTUP: "true"
      PRUNE_CRON: "0 0 4 * * *"
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
    volumes:
      - /root/.ssh/:/root/.ssh/

  sftp_check:
    image: mazzolino/restic
    hostname: wpsftp-check
    environment:
      RUN_ON_STARTUP: "false"
      CHECK_CRON: "0 15 5 * * *"
      RESTIC_CHECK_ARGS: >-
        --read-data-subset=10%
      RESTIC_REPOSITORY: sftp:root@192.168.7.244:/home/new_backup
      RESTIC_PASSWORD: MyPassWord
      TZ: America/Bogota
    volumes:
      - /root/.ssh/:/root/.ssh/

  notify:
    image: mazzolino/apprise-microservice:0.1
    environment:
      NOTIFICATION_URLS: mailtos://sistemas4:mypassword@domain.com.co/sistemas4@domain.com/sistemas2@domain.com?smtp=mail.domain.com.co&port=465&name=Docker%20Backup%20Notification
    networks:
      - notification

networks:
  notification:

Thank you