bit-team / user-callback

Example user-callback scripts
14 stars 9 forks source link

I have written a new callback (example of advanced usage) #7

Open fpoto opened 3 years ago

fpoto commented 3 years ago

user-callback.txt Starting from the code I found here as inspiration, I rewrote it and added some things. I have used it every night for six months now and kept improving and debugging it. Should I post it here as a new file? Make a pull request?

#!/bin/bash

## A user callback for backintime
## Copyright 2021 Fancesco Potortì, released under AGPL 3.0 or later version
##
## Return 0 if everything is alright.
## Any other return code cancels the running snapshot

################################################################
## User variables

PATH=/bin:/sbin:/usr/bin:/usr/sbin

## It is possible to store the backup files on a volume which is mounted before the backup and
## unmounted afterwards.  If this is what you do, then set the following variables
mount_point=            # leave blank if no mounting and unmounting is necessary
mount_point=/backup     # the mount point for storing the backup files
if [ "$mount_point" ]; then
    ## The device to mount for storing the backup files
    mount_dev=          # leave blank if reading device from fstab
    mount_dev=$(/usr/sbin/findfs LABEL=backup)
    ## The options used when mounting the device
    mount_opts=         # leave blank if reading options from fstab
    mount_opts="-o barrier,compress=zstd:15"
fi

## Log
maxlogs=40
## TODO: log options: short or long log, send email, store to /var/log/

################################################################
## Internal variables and code

## Parameters
profile_id="$1"
profile_name="$2"
reason="$3"

local_dir=$HOME/.local/share/backintime/    # where the log file resides
shortlog=$local_dir/shortlog            # short log of the backup process
tmplog=$local_dir/tmplog-$PPID
size_file=$local_dir/init-size-$PPID

((running_under_schedule = (SHLVL == 1)))

function print_free_space () {
    ## Indent df output by two spaces
    df --sync --human-readable --output=target,ipcent,used,avail,pcent $mount_point | sed 's/^/  /'
    echo
}

case $reason in
    1) ## Backup process begins
    if ((running_under_schedule)); then # running under Cron
        rm --force $tmplog
    fi
    ;;
esac

if ((running_under_schedule)); then     # running under Cron
    exec &>>$tmplog     # all output goes to the tmplog file
fi

case $reason in
    1) ## Backup process begins
    date; echo
    echo "- Backup process begins"

    if [ "$mount_point" ]; then
        ## Store initial filesystem size
        df --sync --block-size=1M --output=used $mount_point | tail -1 > $size_file
        print_free_space
    fi
        ;;

    2) ## Backup process ends
    echo "- Backup process ends"

    ## Read takesnapshot_.log, excluding all lines containing 'BACKINTIME' and including
    ## only lines starting with '[I] ', remove the '[I] ' and indent by two spaces
    sed -n '/BACKINTIME/d; /Smart remove/d; /^\[I] /s//  /p' $local_dir/takesnapshot_.log

    if [ "$mount_point" ]; then
        ## Check file system
        echo -n "  Checking file system..."
        mount -o remount,ro $mount_point
        fstype=$(df --output=fstype $mount_point | tail -1)
        device=$(df --output=source $mount_point | tail -1)
        case $fstype in
        btrfs)
            ## --mode lowmem on an SSD 1TB filesystem increases checking time from 8
            ## to 180 minutes and decreases resident memory occupancy from 7 to 5 GB
            #fsck="btrfs check --force --check-data-csum --mode lowmem"
            fsck="btrfs check --force --check-data-csum"
            ;;
        ext*)
            fsck="fsck -T -pf -t ext2"
            ;;
        *)
            fsck="fsck -T"
            ;;
        esac
        chkout=$($fsck $device 2>&1)
        chkret=$?
        if ((chkret == 0)); then
        echo " success"
        else
        if ((chkret == 137)); then
            echo " FAILED: killed by the oom killer"
            echo "Not enough memory available at the moment"
        else
            echo " FAILED: $fsck $device returned $chkret"
            echo $chkout
        fi
        fi

        print_free_space

        ## Compute used space
        isz=$(cat $size_file)
        fsz=$(df --block-size=1M --output=used $mount_point | tail -1)
        echo "  ${fsz}-${isz} = *** $((fsz-isz)) MB used ***"
        rm --force $size_file
    fi

    echo; date
    if ((running_under_schedule)); then # running under Cron
        if command -v mailx &>/dev/null; then
        mailx -s backintime $LOGNAME < $tmplog
        fi
        if command -v savelog &>/dev/null; then
        savelog -lc $maxlogs  $shortlog
        fi
        mv $tmplog $shortlog
    fi
        ;;

    3) ## A new snapshot was taken
        snapshot_id="$4"
        snapshot_name="$5"
    echo -e "- A new snapshot $snapshot_id was taken"
        ;;

    4) ## There was an error
        errorcode="$4"
    echo "- Signaling error code $errorcode"
        case $errorcode in
            1) ## ERROR The application is not configured
        msg="ERROR The application is not configured"
        ;;
            2) ## ERROR A 'take snapshot' process is already running
        msg="ERROR A 'take snapshot' process is already running"
        ;;
            3) ## ERROR Can't find snapshots folder (is it on a removable drive ?)
        msg="ERROR Can't find snapshots folder (is it on a removable drive ?)"
        ;;
            4) ## ERROR A snapshot for 'now' already exist
        msg="ERROR A snapshot for 'now' already exist"
        ;;
        *) ## ERROR Unknown error
        msg="ERROR Unknown error, probably this callback handler is outdated"
        ;;
        esac
    echo "$msg"
        ;;

    5) ## backintime-qt4 (GUI) started
    echo "- GUI started"
        ;;

    6) ## backintime-qt4 (GUI) closed
    echo "- GUI closed"
        ;;

    7) ## Mount drives
    echo "- Mounting drives"
    if [ "$mount_point" ]; then
        echo "Mounting $mount_point"
            umount $mount_point 2> /dev/null || true
        mount $mount_opts $mount_dev $mount_point
    fi
    ;;

    8) ## Unmount the drives
    echo "- Unmounting drives"
    if [ "$mount_point" ]; then
        echo "Unmounting $mount_point"
            umount $mount_point 2> /dev/null || true
    fi
    ;;

    *) ## Unknown reason
    echo "- Unknown callback reason, probably this callback handler is outdated"
    ;;

esac
buhtz commented 2 years ago

Dear @fpoto , apologize for our late reply.

Thanks a lot for your contribution. We will take this into account while restructuring our documentation.

fpoto commented 2 years ago

Just for the record, since I sent the code one year ago, I have kept using it every night without problems. For me, the code is stable. If anyone has suggestions, I'd be glad to hear from them.

buhtz commented 3 weeks ago

Hello Francesco, we won't forget you and your contribution.

https://github.com/bit-team/backintime/issues/1838

Can you please add some more description. It is not clear to me what your script is doing (and I hate to read bash/sh) and why it should be added as an example script. The question is if your script is a helpful example for other users. My first impression is that it is quit complex for an "example" and that it is doing much more then just one thing.

fpoto commented 3 weeks ago

Basically it is just a wrapper around a night backup called by cron or a manual backup invoked by the gui.

If the backups are kept on a separate partition which is only mounted at backup time, it checks whether it is already mounted and mounts it otherwise, remounts it read-only when it's finished, checks the filesystem and computes the total space used by the backup and the remaining space on the file system

Checks for several errors, including out-of-memory, and reports them on the log of what's happened (mounting, space available before and after backup, possible errors, unmounting).

When called from cron, it emits logging info on the standard output, which cron then emails to the caller.

That's it. If you want, I can write down a more formal list of features and usage.

buhtz commented 3 weeks ago

Thank you for your reply.

I would say your script is not a "regular" example usable for Back In Time (BIT) beginners to learn how user-callback works. It is to complex for this. But I can add it as a real-world example to illustrate how user-callback can be use on an advanced level. It would be great if you could improve the in-code comments in your script and also improve the introducing comments in the beginning to increase the outcome for new users reading the user-callback docu.

btw: I'll soon migrate the user-callback docu and the example scripts into bit-team/backintime repo.

fpoto commented 3 weeks ago

I think my script is not an example for beginners, but a generic complete callback that is really usable in practice by anyone without modification.

What it does:

It is a real-world example which is usable in practice as it is, but I would not call it advanced.

I hope I'll find the time to improve the documentation. Maybe for a start the above explanation would be better than nothing.

Thank you for caring about BIT and my contribution

-- Francesco Potortì (ricercatore) ISTI - CNR, Pisa, Italy
Web: http://fly.isti.cnr.it Skype: wnlabisti Mobile: +39.348.8283.107 also Telegram, and even Wa

buhtz commented 1 week ago

Thank you very much for your efforts.

I am migrating the user-callback docu und examples into "backintime" repo. I will close the issue and setting its repo into "archive mode" in the foreseen future.

I would suggest that you open a PR or Issue with your improved script. Might it be an idea to integrate it directly into the documentation instead of adding it as an example script beside the others?

It would be nice if you could rethink your license of that code. We will use your script in the documentation and/or as an example. Other users might use it or just code snippets of it. Or they use at as in inspiration for their own scripts. With all respect to your work I won't like to bother users with (complex) licensing stuff. A regular user might be insecure about if he/she is allowed to use that code or not. And a user also might not be willing to dive into the topic about licensing to answer this question to him/herself. Again: With all respect to your work it is just a script and not a full application. So I would like to keep it simple for the users. I would suggest CC0-1.0 (public domain) as a "license".

Otherwise it is no big deal if you like to stick to AGPL. This won't be a blocker.

Best, Christian

buhtz commented 1 week ago

PR backintime#1899