Can you explain the restore feature better?

[kevdogg]

So I think I have everything about working however I'm really stuck on the restore process.

Here is my docker-compose service:

  bitwarden-backup:
    container_name: bitwarden-backup
    hostname: bitwarden-backup
    image: ttionya/vaultwarden-backup:latest
    restart: always
    networks:
      - bitwarden_rs_net
    secrets:
      - heirloom-mail-x_secret
    depends_on:
      <<: *bitwarden-depends
    volumes:
      - /home/bitwarden/.config/vaultwarden-backup:/config
      - /var/data/bw-data:/data
    environment:
      - RCLONE_REMOTE_NAME=arch-tm
      - RCLONE_REMOTE_DIR=/mnt/bitwarden-backup
      - CRON=*/5 * * * *
      - ZIP_ENABLE=TRUE
      - ZIP_TYPE=7z
      - BACKUP_KEEP_DAYS=30
      - BACKUP_FILE_DATE_SUFFIX=-%r
      - TIMEZONE=${TZ}
      - MAIL_SMTP_ENABLE=TRUE
      - MAIL_SMTP_VARIABLES_FILE=/run/secrets/heirloom-mail-x_secret
      - MAIL_WHEN_SUCCESS=TRUE
      - MAIL_WHEN_FAILURE=TRUE
      - DATA_DIR=/data

My config file is the following:

[arch-tm]
type = sftp
host = arch-tm.domain.com
user = bitwarden
key_file = ${RCLONE_CONFIG_DIR}/merged_key
pubkey_file = ${RCLONE_CONFIG_DIR}/merged_key
known_hosts_file = ${RCLONE_CONFIG_DIR}/known_hosts
md5sum_command = md5sum
sha1sum_command = sha1sum

On the remote server (the target of the backups), I have directory listings similar to the following:

'backup.20220304-01:10:00 AM.7z'  'backup.20220304-12:51:00 AM.7z'
'backup.20220304-01:15:00 AM.7z'  'backup.20220304-12:52:00 AM.7z'
'backup.20220304-01:20:00 AM.7z'  'backup.20220304-12:53:00 AM.7z'
'backup.20220304-01:25:00 AM.7z'  'backup.20220304-12:54:00 AM.7z'
'backup.20220304-12:46:00 AM.7z'  'backup.20220304-12:55:00 AM.7z'
'backup.20220304-12:47:00 AM.7z'  'backup.20220304-12:56:00 AM.7z'
'backup.20220304-12:48:00 AM.7z'  'backup.20220304-12:57:00 AM.7z'
'backup.20220304-12:49:00 AM.7z'  'backup.20220304-12:58:00 AM.7z'
'backup.20220304-12:50:00 AM.7z'  'backup.20220304-12:59:01 AM.7z'

Extracting one of the backups manually I get the following:

'config.20220304-01:10:00 AM.json'  'rsakey.20220304-01:10:00 AM.tar'
'db.20220304-01:10:00 AM.sqlite3'

So if I wanted manually to restore the backup -- I'd stop the vaultwarden container, and I could copy config.20220304-01:10:00 AM.json->config.json, db.20220304-01:10:00 AM.sqlite3->db.sqlite3, and manually extract the rsakeys and also restore all those.

I'm not sure however how to reverse the process. The example given in the documentation I didn't understand.

  --mount type=volume,source=vaultwarden-data,target=/bitwarden/data/ \
  --mount type=bind,source=$(pwd),target=/bitwarden/restore/ 

I don't see how the backup is going to restore over sftp -- I don't see a config file here anywhere this is mounted.
Could you help with maybe more examples?

Oh one last thing although the mail test works, when trying to send mail from the program -- I'm getting this in my logs:

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2022-03-04 03:00:01 ....A         1377        35008  config.20220304-03:00:01.json
2022-03-04 03:00:01 ....A       249856               db.20220304-03:00:01.sqlite3
2022-03-04 03:00:01 ....A         7680               rsakey.20220304-03:00:01.tar
------------------- ----- ------------ ------------  ------------------------
2022-03-04 03:00:01             258913        35008  3 files
upload backup file to storage system
delete 30 days ago backup files
mail send was successfully
send-mail: applet not found

No mail is actually sent when transfer is completed. successfully.

[ttionya]

Hey @kevdogg ,

The restore instructions are indeed not clearly described, I tried to update it.

--mount type=bind,source=$(pwd),target=/bitwarden/restore/

It will map the current directory to the container's /bitwarden/restore, directory where the restore program will look for backup files.

--mount type=volume,source=vaultwarden-data,target=/bitwarden/data/ \

Depending on your docker-compse.yml, it doesn't apply to you, for you you can use --mount type=bind,source=/var/data/bw-data/,target=/data/. It is used to specify where the files are restored to.

For email, do you specify the MAIL_TO environment variable?

[kevdogg]

Ok so looking at your instructions let me kind of go through the restore process with you to see if I make sense. Per the backups, I'm using rclone via sftp to transfer the backup images password protected via a 7zip archive to a remote computer. In order to restore, Id first have to select my specific archive, and actually manually transfer this archive back to the bitwarden host. I'll have to unzip the archive to create a directory, and then use the appropriate bind parameter to link the local directory to the container. I guess I was under the impression the actual rclone command would be reversed -- however this isn't the case since this requires some manual intervention to bring back the password protected archive and unzip the archive.

My original bind volumes were mounted into the container via:

• /var/data/bw-data:/data

So lets say I'm currently in a local directory that has been unzipped containing the backup sources I want to restore, I suppose the command would be:

docker run --rm -it \ --mount type=bind,source=var/data/bw-data/,target=/data/ \ --mount type=bind,source=$(pwd),target=/bitwarden/restore/ \ -e DATA_DIR="/data" \ ttionya/vaultwarden-backup:latest restore

In terms of the mail issue - I think I got it figured out. Honestly the heirloom mailx program that is used -- there isn't a ton of examples out there on the internet -- I found more using the actual mail program (which i think is actual snail) and less on heirloom mailx. A lot of the sites out there don't distinguish between the two so syntax is tricky since its not the same.

What I did in my compose file was use the following:

        - MAIL_TO=<user>@gmail.com
        - MAIL_SMTP_ENABLE=TRUE
        - MAIL_SMTP_VARIABLES_FILE=/run/secrets/heirloom-mail-x_secret
        - MAIL_WHEN_SUCCESS=TRUE
        - MAIL_WHEN_FAILURE=TRUE

With the secrets file I have the following: -S smtp=smtps://postfix.<domain>.com:465 -S smtp-auth=login -S smtp-auth-user=bitwarden -S smtp-auth-password=<password> -S ssl-verify=strict -r bitwarden@<domain>.com(Bitwarden-Backup@archbw)

I used the -r parameter although I've seen other documentation use the -S sender= parameter. Anyway the part contained in the () can not have any spaces -- that was part of the problem --- no spaces within the parentheses. The regular mail program would use different synxtax.

Lastly, unrelated to the two above topics The container looks like it needs to be run as root. I tried running the container as a non root user, however the cron timers and such couldn't be set up as a non-root user. I haven't looked at the code per say -- but some of the Dockerfiles, but I don't know why a crontab couldn't be set up as a non-root user and run that way.

Thanks for reply.

[ttionya]

Thanks for the detailed explanation.

Now the container cannot be started as a non-root user, because crond can only be started as root user. I remembered that there was feedback about this before and I will try to find a way to support this feature.

Back to the main topic. The tool's restore is designed to be rclone-independent, no rclone configuration or rclone environment variables are required, just specify the files to be restored and map them to the container's specified directory. So you need to manually download the backup files from the remote storage to the computer that needs to be restored. And you don't need to unzip the files manually, just use --zip-file to specify the backup zip file.

Because the restore operation is hardly ever used, and to do it automatically by rclone to download the files from the remote storage and restore them, you need to pass multiple environment variables (like RCLONE_REMOTE_NAME and RCLONE_REMOTE_DIR), which increases the configuration cost. So I need to think if it is worth to support rclone based restore.

About mail, I don't know much about linux using SMTP to send mail, I chose "heirloom mailx" just because I happened to find it at that time and tested that I could send mail properly. If you don't use the -r parameter, but use -S from=address will it work properly?

[kevdogg]

So if I use the --zip-file argument, does the process prompt for a password? I ask this because the password is usually kept within the config directory and I noticed that the restore command (if run in isolation) does not require the config volume to be mounted.

[ttionya]

@kevdogg ,

So if I use the --zip-file argument, does the process prompt for a password?

Yes, if you do not specify a password with -p/--password, the password will be asked for interactively.

[kevdogg]

Hey thanks for that information. Are there any other hidden flags for the restore command? I only ask this since the password flag wasn't included in the documentation?

[ttionya]

@kevdogg ,

All environment variables and parameters are listed in the documentation. The restored option is just collapsed, see restore options.

[marklockwood-techie]

I too am having problems running a restore.

I backup using the automatic method, and my Vaultwarden data is mapped locally to /bw-data/

I've downloaed the zipped backup locally to /restore (putting the data to /restore/restore/backup.20220908.zip)

I run; cd /restore

docker stop bitwarden

docker run --rm -it \ --mount type=bind,source="/bw-data/",target=/data/ \ --mount type=bind,source=$(pwd),target=/bitwarden/restore/ \ -e DATA_DIR="/data" \ ttionya/vaultwarden-backup:latest restore -zip-file

And it returns Empty input

What am I missing? Thanks

[ttionya]

@marklockwood-techie ,

The tool does not automatically use any files, so you need to specify the files to be restored.

So

...
ttionya/vaultwarden-backup:latest restore --zip-file <file>

Maybe the documentation is not clear, I will optimize it.

[marklockwood-techie]

Yes, that works now, thanks again