Isssue with starting containers on Synology

1kohm commented 1 year ago

Hi,

thanks for the containers, it's something that I'm looking for to backup my local vaultwarden instance but I'm having problem with understanding how to acually setup that. Here's what I've done so far:

setup rclone with "docker run --rm -it --mount type=volume,source=vaultwarden-rclone-data,target=/config/ ttionya/vaultwarden-backup:latest rclone config"
Result of rclone requested with "docker run --rm -it --mount type=volume,source=vaultwarden-rclone-data,target=/config/ ttionya/vaultwarden-backup:latest rclone config show" comamnd:

[BitwardenBackup] type = smb host = 10.0.0.130 user = rclone pass = secretpasshashthatisprivate

Instalation was done with docker.compose.yml file (see config below)

{
version: '3.4'

services:

  vaultwarden:
    image: vaultwarden/server:latest
    container_name: vaultwarden
    restart: always
    environment:
      WEBSOCKET_ENABLED: "true"  # Enable WebSocket notifications.
      LOG_FILE: /data/vaultwarden.log
      INVITATIONS_ALLOWED: false
      SIGNUPS_ALLOWED: true
      EXTENDED_LOGGING: true
      LOG_LEVEL: trace
      PASSWORD_ITERATIONS: 700000
    #   ADMIN_TOKEN: 'your authentication token'
    ports:
     # - '127.0.0.1:8200:80'
     - 801:80    
     - 3012:3012
    volumes:
      - ./vw-data:/data

  backup:
    image: ttionya/vaultwarden-backup:latest
    restart: always
    environment:
    #   RCLONE_REMOTE_NAME: 'BitwardenBackup'
    #   RCLONE_REMOTE_DIR: '/BitwardenBackup/'
    #   RCLONE_GLOBAL_FLAG: ''
    #   CRON: '5 * * * *'
    #   ZIP_ENABLE: 'TRUE'
    #   ZIP_PASSWORD: 'WHEREISMYPASSWORD?'
    #   ZIP_TYPE: 'zip'
    #   BACKUP_FILE_DATE_SUFFIX: ''
    #   BACKUP_KEEP_DAYS: 0
    #   PING_URL: ''
    #   MAIL_SMTP_ENABLE: 'FALSE'
    #   MAIL_SMTP_VARIABLES: ''
    #   MAIL_TO: ''
    #   MAIL_WHEN_SUCCESS: 'TRUE'
    #   MAIL_WHEN_FAILURE: 'TRUE'
    #   TIMEZONE: 'UTC'
    volumes:
      - ./vw-data:/bitwarden/data/
      - ./vaultwarden-rclone-data:/config/
    #   - /path/to/env:/.env

volumes:
  vaultwarden-data:
    # Specify the name of the volume where you save the vaultwarden data,
    # use vaultwarden-data for new users
    # and bitwardenrs-data for migrated users
    name: vaultwarden-data
    # name: bitwardenrs-data
  vaultwarden-rclone-data:
    external: true
    # Specify the name of the volume where you save the rclone configuration,
    # use vaultwarden-rclone-data for new users
    # and bitwardenrs-rclone-data for migrated users
    name: vaultwarden-rclone-data
    # name: bitwardenrs-rclone-data
}

Vaultwarden is running fine with that configuration but ttionya/vaultwarden-backup:latest container is failing to start and complaining that rclone is not configured correctly.

So what I am doing incorrectly?

ttionya commented 1 year ago

The command docker run --rm -it --mount type=volume,source=vaultwarden-rclone-data,target=/config/ ttionya/vaultwarden-backup:latest rclone config is used to generate an rclone configuration file, which is stored in the vaultwarden-rclone-data volume.

Please take a look at the docker-compose.yml file:

services:
  backup:
    volumes:
      - ./vaultwarden-rclone-data:/config/ # remove this line

Since you have already mapped the local directory ./vaultwarden-rclone into the container, you need remove this line and backup will work.

1kohm commented 1 year ago

Hi @ttionya, thanks for the tip. I've removed " - ./vaultwarden-rclone-data:/config/" from volumes (see the config below), recreated the containers but "ttionya/vaultwarden-backup:latest" still can't find the rclone configuration file that should be in "/config/" folder on container itself, as it is, right?

  backup:
    image: ttionya/vaultwarden-backup:latest
    restart: always
   # environment:
    #   RCLONE_REMOTE_NAME: 'BitwardenBackup'
    #   RCLONE_REMOTE_DIR: '/BitwardenBackup/'
    #   RCLONE_GLOBAL_FLAG: ''
    #   CRON: '5 * * * *'
    #   ZIP_ENABLE: 'TRUE'
    #   ZIP_PASSWORD: 'WHEREISMYPASSWORD?'
    #   ZIP_TYPE: 'zip'
    #   BACKUP_FILE_DATE_SUFFIX: ''
    #   BACKUP_KEEP_DAYS: 0
    #   PING_URL: ''
    #   MAIL_SMTP_ENABLE: 'FALSE'
    #   MAIL_SMTP_VARIABLES: ''
    #   MAIL_TO: ''
    #   MAIL_WHEN_SUCCESS: 'TRUE'
    #   MAIL_WHEN_FAILURE: 'TRUE'
    #   TIMEZONE: 'UTC'
    volumes:
      - ./vw-data:/bitwarden/data/
     # - ./vaultwarden-rclone-data:/config/
    #   - /path/to/env:/.env

volumes:
  vaultwarden-data:
    # Specify the name of the volume where you save the vaultwarden data,
    # use vaultwarden-data for new users
    # and bitwardenrs-data for migrated users
    name: vaultwarden-data
    # name: bitwardenrs-data
  vaultwarden-rclone-data:
    external: true
    # Specify the name of the volume where you save the rclone configuration,
    # use vaultwarden-rclone-data for new users
    # and bitwardenrs-rclone-data for migrated users
    name: vaultwarden-rclone-data
    # name: bitwardenrs-rclone-data

1kohm commented 1 year ago

Hi, I found my mistake :) the problem was that I didn't understood the difference between two types of "volumes". There're named volumes and path volumes. My issue was that I've used path volume that's indicated with "./" instead of named one. Here's the correct config that worked:

services:
  backup:
    volumes:
      - ./vw-data:/bitwarden/data/
      - vaultwarden-rclone-data:/config/

volumes:
  vaultwarden-rclone-data:
    external: true
    name: vaultwarden-rclone-data

Next question is how to configure the container environments variable to save backup files outside container in fileshare?

ttionya commented 1 year ago

This backup tool is based on rclone, and you can refer to the rclone documentation to confirm if it supports backing up to fileshare.

1kohm commented 1 year ago

Hi,

now I can confirm that backup to smb with rclone wroks :) I'm able to get it automatically backup via cron and files are being copied.

The last issue that I'm facing with is the restore. Running that commands

  docker run --rm -it \
  --mount type=bind,source=/volume2/docker/vaultwarden-backup/vw-data/,target=/data/ \
  --mount type=bind,source=$(pwd),target=/bitwarden/restore/ \
  -e DATA_DIR="/data" \
  ttionya/vaultwarden-backup:latest restore --zip-file /volume2/docker/vaultwarden-backup/backup/backup.zip

Output that logs but when I login into vaultwarden then database is actually empty :( How to troubleshoot that?

(Default: n): y
restore vaultwarden backup zip file

7-Zip (z) 22.01 (x64) : Copyright (c) 1999-2022 Igor Pavlov : 2022-07-15
 64-bit locale=C.UTF-8 Threads:4

Scanning the drive for archives:
1 file, 331483 bytes (324 KiB)

Extracting archive: /bitwarden/restore/backup.20230408.zip
--
Path = /bitwarden/restore/backup.20230408.zip
Type = zip
Physical Size = 331483

Enter password:secret

Everything is Ok

Files: 4
Size:       994816
Compressed: 331483
extract vaultwarden backup zip file successful
restore vaultwarden sqlite database
restore vaultwarden sqlite database successful
restore vaultwarden rsakey
restore vaultwarden rsakey successful
restore vaultwarden attachments
restore vaultwarden attachments successful
restore vaultwarden sends
restore vaultwarden sends successful

ttionya commented 1 year ago

If you are using docker-compose, you do not need to specify the DATA_DIR environment variable, remove it.

1kohm commented 1 year ago

my sqlite database keeps failing after restarting container even without your container running.

[panic][ERROR] thread 'rocket-worker-thread' panicked at 'Error loading ciphers: DatabaseError(Unknown, "database disk image is malformed")': src/db/models/cipher.rs:633 So I'm trying different approach using mariadb, I got it running now, backup job with your container seems to create db.20230409.sql file but in your documentation point 3 in Restore section looks incomplete. How I can test if restore is successful?

https://github.com/ttionya/vaultwarden-backup/blob/master/docs/using-the-mysql-or-mariadb-backend.md

ttionya commented 1 year ago

I cannot ascertain whether it is due to the backup tool or vaultwarden, but I came across an issue dani-garcia/vaultwarden#2851 related to Vaultwarden.

Perhaps, I should start considering ways to ensure the availability of backup files.

However, my familiarity with databases is limited, and with my recent workload being heavy, progress in this regard may be slow.

1kohm commented 1 year ago

thanks for the input @ttionya, i really appreciate it. I belive that crashing sqlite database is due to corrupted WAL files and the fact that transactions are still being open while executing the backup. That's the reason I'm planning to switch to more reliable mariadb, at least I hope :)

In regards to database restore I'm also a noob but I can at least confirm that executing that command will successfully restore mariadb database backup with your script. Only thing what would be great to add is that it could also restore whole vaultwarden /data directory. I think that shouldn't be diffiicult to write "restore" script that would do it all at once.

docker exec -i db_container_name mysql [--user yourusername] [--password=yourpassword] databasename < /path/to/db.dump

ttionya commented 1 year ago

I apologize for not comprehending your meaning. The current restore script is restore the backup files to the /data directory.

ttionya / vaultwarden-backup

Isssue with starting containers on Synology #107