search

OpenMediaVault-Plugin-Developers / openmediavault-borgbackup

openmediavault plugin for borgbackup
10 stars 5 forks source link

Pre/post-scripts not being run on backups triggered manually via OMV web UI #41

Closed kamild1996 closed 5 months ago

kamild1996 commented 1 year ago

I would like to use the pre/post-script feature for the exact same purpose as mentioned in #33, but it seems like the plugin is ignoring it? Inside my archive I set:

but when the backup is run, I see no mention at all about possible results of these commands, and the docker service is not being stopped.

Am I not using this right? openmediavault 6.1.4-2, openmediavault-borgbackup 6.4.2

ryecoaaron commented 1 year ago

Did you look to see if the borg scripts in /var/lib/openmediavault/borgbackup/ have your commands in them? The commands should be - https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-borgbackup/blob/master/srv/salt/omv/deploy/borgbackup/default.sls#L156 and https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-borgbackup/blob/master/srv/salt/omv/deploy/borgbackup/default.sls#L194

kamild1996 commented 1 year ago

Did you look to see if the borg scripts in /var/lib/openmediavault/borgbackup/ have your commands in them? The commands should be - https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-borgbackup/blob/master/srv/salt/omv/deploy/borgbackup/default.sls#L156 and https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-borgbackup/blob/master/srv/salt/omv/deploy/borgbackup/default.sls#L194

Looking at the scripts inside of that directory, I can see my commands in their respective sections.

What I have also just noticed is that according to these scripts, there should be a log file in /var/log/borgbackup.log, but when I run the backup, it's not being created.

ryecoaaron commented 1 year ago

I use this plugin myself and my log files are extensive. I'm not sure why your system isn't creating them. Have you tried executing one of the scripts as root and looking for error messages and/or non-zero exit codes?

kamild1996 commented 1 year ago

I use this plugin myself and my log files are extensive. I'm not sure why your system isn't creating them. Have you tried executing one of the scripts as root and looking for error messages and/or non-zero exit codes?

I believe they are already being executed as root, considering the files within the repo are owned by root.

Here's the full log I get after running the backup from inside OMV (except for the file list, which I trimmed):

Command: export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; export BORG_PASSPHRASE=''; export OMV_BORGBACKUP_STARTING_HOUR="1"; /usr/bin/borg create --list --stats --verbose --compression auto,lzma,9 "/srv/dev-disk-by-uuid-862e1e8e-e766-44d4-8d44-e04e506bcb81/Backup/docker/::Docker-{now:%Y-%m-%d_%H-%M-%S}" '/var/lib/docker/volumes' --exclude '' 2>&1

Creating archive at "/srv/dev-disk-by-uuid-862e1e8e-e766-44d4-8d44-e04e506bcb81/Backup/docker/::Docker-2023-01-09_19-36-34"
b /var/lib/docker/volumes/backingFsBlockDev
U /var/lib/docker/volumes/portainer_data/_data/certs/cert.pem
U /var/lib/docker/volumes/portainer_data/_data/certs/key.pem
[...]
d /var/lib/docker/volumes
------------------------------------------------------------------------------
Repository: /srv/dev-disk-by-uuid-862e1e8e-e766-44d4-8d44-e04e506bcb81/Backup/docker
Archive name: Docker-2023-01-09_19-36-34
Archive fingerprint: cf1eadfa7366668f5ad60ed120e181bf7076094a1373c5ccba380b54bc6adfda
Time (start): Mon, 2023-01-09 19:36:34
Time (end):   Mon, 2023-01-09 19:36:35
Duration: 0.83 seconds
Number of files: 484
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               15.69 MB              8.27 MB            153.68 kB
All archives:               31.27 MB             16.52 MB              8.40 MB

                       Unique chunks         Total chunks
Chunk index:                     424                  861
------------------------------------------------------------------------------

END OF LINE
█
ryecoaaron commented 1 year ago

I wrote the plugin. I am aware they run as root. I wanted you to run from the command and was making sure that you would run it as root. Running from the web interface could potentially not show some output and you couldn't check the exit code (echo $?). What is the output of: ls -al /var/log/borgbackup.log*

kamild1996 commented 1 year ago

I misread and didn't quite catch your request, sorry about that.

When I run the backup script directly from shell, it seems to be working correctly, including running the pre/post-launch commands and generating the log file. So I assume these tasks aren't being done only when running the backup manually from the OMV web panel.

root@piec:~# /var/lib/openmediavault/borgbackup/daily.d/borgbackup-5fe4e6e7-7b42-4bb1-8196-fb08e35f0be2

Mon 09 Jan 2023 08:11:59 PM CET Executing pre-script

Warning: Stopping docker.service, but it can still be activated by:
  docker.socket

Mon 09 Jan 2023 08:12:04 PM CET Starting backup

Creating archive at "/srv/dev-disk-by-uuid-862e1e8e-e766-44d4-8d44-e04e506bcb81/Backup/docker/::Docker-2023-01-09_20-12-04"
M /var/lib/docker/volumes/portainer_data/_data/portainer.db
A /var/lib/docker/volumes/bitwarden_data/_data/db.sqlite3-wal
M /var/lib/docker/volumes/bitwarden_data/_data/vaultwarden.log
M /var/lib/docker/volumes/syncthing_config/_data/index-v0.14.0.db/LOG
M /var/lib/docker/volumes/syncthing_config/_data/index-v0.14.0.db/000243.log
------------------------------------------------------------------------------
Repository: /srv/dev-disk-by-uuid-862e1e8e-e766-44d4-8d44-e04e506bcb81/Backup/docker
Archive name: Docker-2023-01-09_20-12-04
Archive fingerprint: ee7f8a34a8656a4ed086d18741f4031173490e9bd81fb7d13b2eacb49a3afd1b
Time (start): Mon, 2023-01-09 20:12:04
Time (end):   Mon, 2023-01-09 20:12:05
Duration: 0.46 seconds
Number of files: 484
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               15.69 MB              8.27 MB            158.34 kB
All archives:               46.96 MB             24.79 MB              8.59 MB

                       Unique chunks         Total chunks
Chunk index:                     431                 1292
------------------------------------------------------------------------------
terminating with success status, rc 0

Mon 09 Jan 2023 08:12:05 PM CET Executing post-script

Mon 09 Jan 2023 08:12:08 PM CET Pruning repository

Keeping archive (rule: daily #1):        Docker-2023-01-09_20-12-04           Mon, 2023-01-09 20:12:04 [ee7f8a34a8656a4ed086d18741f4031173490e9bd81fb7d13b2eacb49a3afd1b]
Pruning archive (1/1):                   Docker-2023-01-09_19-36-34           Mon, 2023-01-09 19:36:34 [cf1eadfa7366668f5ad60ed120e181bf7076094a1373c5ccba380b54bc6adfda]
Keeping archive (rule: daily[oldest] #2): Docker-2023-01-09_19-27-54           Mon, 2023-01-09 19:27:54 [4233a1c9a0bd642e5b9c05ebd05e538977a83ed444b63addd948a161cab84f63]
terminating with success status, rc 0
root@piec:~# ls -al /var/log/borgbackup.log*
-rw-r--r-- 1 root root 2219 Jan  9 20:12 /var/log/borgbackup.log
ryecoaaron commented 1 year ago

Due to the web interface needing to capture output, it is possible the manual execution from the web interface might not log. But the plugin's script is meant to run from cron which should be the same as running manually from the command line as root.

kamild1996 commented 1 year ago

I see, thank you for explanation. Just verified everything seems to works properly when a scheduled backup runs.

riker09 commented 1 year ago

Hello. I just encountered the same thing: Running the archive from the Web GUI gives me some output in the window, but it does not create a log file in /var/log/borgbackup.log nor does it trigger my script. This seems somewhat counter-intuitive in my opinion. Why is there a difference when executing from the web gui? At least this should be documented in below the popup window.

What kind of information is available when running the pre/post script? I'd like to get notified via my own NTFY instance, so I use curl:

curl --header 'X-Tags: white_check_mark' --data 'Backup susccessful' https://ntfy.sh/my-backup-topic

Is it possible to include the name of the Archive job as an ENV var? --data "Backup of ${NAME_HERE} successful" Can I differentiate between a successful and unsuccessful job? Guess that will be the exit code?

ryecoaaron commented 1 year ago

Running the archive from the Web GUI gives me some output in the window, but it does not create a log file in /var/log/borgbackup.log

It is just supposed to be a test run. It literally only run the borg command not the entire script that the cron job runs. Since it isn't an hourly, daily, weekly, monthly, or yearly run, the script shouldn't be run because it would purge a true timed archive depending on which script was selected. And since it can either log to a file or to the window, I chose to have it log to the window. If you need to save that output, I guess you could cut and paste it.

nor does it trigger my script.

It is only running the borg command right now but I guess I could change it to run the pre and post scripts if either field is populated.

Why is there a difference when executing from the web gui? At least this should be documented in below the popup window.

I hate documentation on the window because then I am the only one really maintaining it and it looks too busy. I agree it should be in the docs for the borg plugin which are very minimal right now but that is being worked on. These docs are linked to from the plugin but the wiki doesn't exist yet.

What kind of information is available when running the pre/post script?

Any environment variable set in the Environment Variables tab for that specific repo or anything you pass in the pre/post field. If you want --data in it, just put it in the pre or post field. It is just exec'ing whatever you put in there.

Can I differentiate between a successful and unsuccessful job? Guess that will be the exit code?

Yep, as normal, 0 is successful and non-zero is not.

ryecoaaron commented 1 year ago

By the way, here are the best docs for the plugin currently - https://forum.openmediavault.org/index.php?thread/36602-how-to-make-backups-with-borg-using-borgbackup-plugin/&postID=256197&highlight=borg#post256197

ryecoaaron commented 1 year ago

Pre and post script will be run when using Run in Archives tab in next release https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-borgbackup/commit/1b32c4dba86f95ba52b818fb7b6ce9aeba524fea

riker09 commented 1 year ago

I did not expect such a quick response. Sorry when I came off a bit harsh earlier. That was not my intention. I do appreciate all your hard work on this awesome plugin. I have already discovered the linked forum documentation and this is what got me to a working state after all, so it has been very helpful. Only the fact that there is a difference between real execution of the script and triggering it via gui was not clear to me. Better documentation and changes are coming, so I'll just be patient. 🙂