Incremental resolv with symbolic link possible?

[chrenderle]

Hello,

I want to create daily incremental raw backups where the parent is the latest on a backup drive which is not connected. The raw backup can then be uploaded to the cloud thus creating daily backups without the need to have the backup drive present and not uploading a full backup to the cloud.

In my example I have the following drives: -main drive: crypt_normal -backup drive: crpyt_backup_1

And the following folder structure: /mnt/crypt_normal/@Applications - main subvolume /mnt/crypt_normal/.snapshots - snapshots directory /mnt/crypt_normal/.crypt_backup_1 - contains symbolic link like this: ../.snapshots/@Applications.Timestamp /mnt/crypt_backup_1 - target directory for snapshots on this drive

The idea is to write a script which, after every backup to crpyt_backup_1, creates a folder (.crypt_backup_1) in crypt_normal with a symbolic link pointing to the latest (or all) snapshot on the backup drive. When I create the daily backups it should look in the folder with the symbolic links for parents to use for the incremental backup.

When I run a dryrun of the following config I get no action:

imestamp_format        long
transaction_log            /var/log/btrbk.log

snapshot_preserve_min   30d
snapshot_preserve       60d
snapshot_create         no
snapshot_dir    .crypt_backup_1

volume /mnt/crypt_normal
        target raw /home/admin/btrbk-test
        incremental_resolve directory
        subvolume @Applications

I guess the reason is that it doesn't see the existing snapshot (via the symbolic link) and thus takes no action and (if not disabled) creates a new one. I think btrbk doesn't follows symbolic links for snapshots. Since the btrfs progs can do this (tested via btrfs su show) I don't think it would be to hard to implement this. The best solution would be that btrbk saves which snapshots are present on which target and can create an incremental snapshot on this basis (if the parent snapshot is present on the snapshot_dir), but this would probably be more work than following symbolic links.

I hope I explained everything clearly. Otherwise I can clarify any questions.

Debug output:

btrbk command line client, version 0.32.1  (Wed Mar 30 17:19:24 2022)
Using configuration: /etc/btrbk/btrbk-test.conf
Initializing volume section: /mnt/crypt_normal
Initializing subvolume section: /mnt/crypt_normal/@Applications
Resolving mount point for: /mnt/crypt_normal/@Applications
### cat /proc/self/mountinfo
Command execution successful
### readlink -v -e '/mnt/crypt_normal/@Applications'
Command execution successful
Real path for "/mnt/crypt_normal/@Applications" is: /mnt/crypt_normal/@Applications
Mount point for "/mnt/crypt_normal/@Applications": /mnt/crypt_normal (mount_source=/dev/bcache0, fs_type=btrfs)
### btrfs subvolume list -a -c -u -q -R '/mnt/crypt_normal'
Command execution successful
Parsed 176 total subvolumes for filesystem at: /mnt/crypt_normal
### btrfs subvolume list -a -r '/mnt/crypt_normal'
Command execution successful
Parsed 168 readonly subvolumes for filesystem at: /mnt/crypt_normal
### btrfs subvolume show --rootid=5 '/mnt/crypt_normal'
Command execution successful
Parsed 11 subvolume detail items: /mnt/crypt_normal
Found btrfs root: /mnt/crypt_normal
Resolving mount point for: /mnt/crypt_normal/.crypt_backup_1
### readlink -v -e '/mnt/crypt_normal/.crypt_backup_1'
Command execution successful
Real path for "/mnt/crypt_normal/.crypt_backup_1" is: /mnt/crypt_normal/.crypt_backup_1
Mount point for "/mnt/crypt_normal/.crypt_backup_1": /mnt/crypt_normal (mount_source=/dev/bcache0, fs_type=btrfs)
Initializing target section: /home/admin/btrbk-test
Creating raw subvolume list: /home/admin/btrbk-test
### find '/home/admin/btrbk-test' -maxdepth 1 -type f -name \*.btrfs.\*info -exec echo INFO_FILE=\{\} \; -exec cat \{\} \;
Command execution successful
Parsed 0 raw info files in path: /home/admin/btrbk-test
Found 0 raw subvolume backups in: /home/admin/btrbk-test
Evaluating snapshot creation for: /mnt/crypt_normal/@Applications
Snapshot creation disabled (snapshot_create=no)
Found 0 related snapshots of "/mnt/crypt_normal/@Applications" in: /mnt/crypt_normal/.crypt_backup_1/@Applications.*
Checking for missing backups of subvolume "/mnt/crypt_normal/@Applications" in "/home/admin/btrbk-test/"
No missing backups found
Found 0 related snapshots of "/mnt/crypt_normal/@Applications" in: /mnt/crypt_normal/.crypt_backup_1/@Applications.*
Cleaning backups of subvolume "/mnt/crypt_normal/@Applications": /home/admin/btrbk-test/@Applications.*
Schedule: preserving all forever
Scheduler reference time: 2022-03-30T17:19:24+0200
Preserving 0/0 items
Deleted 0 subvolumes in: /home/admin/btrbk-test/@Applications.*
Cleaning snapshots: /mnt/crypt_normal/.crypt_backup_1/@Applications.*
Schedule: preserving all within 30 days; first of day (starting at 00:00) for 60 days
Scheduler reference time: 2022-03-30T17:19:24+0200
Preserving 0/0 items
Deleted 0 subvolumes in: /mnt/crypt_normal/.crypt_backup_1/@Applications.*
Completed within: 0s  (Wed Mar 30 17:19:24 2022)
--------------------------------------------------------------------------------
Backup Summary (btrbk command line client, version 0.32.1)

    Date:   Wed Mar 30 17:19:24 2022
    Config: /etc/btrbk/btrbk-test.conf
    Dryrun: YES

Legend:
    ===  up-to-date subvolume (source snapshot)
    +++  created subvolume (source snapshot)
    ---  deleted subvolume
    ***  received subvolume (non-incremental)
    >>>  received subvolume (incremental)
--------------------------------------------------------------------------------
/mnt/crypt_normal/@Applications
<no_action>

NOTE: Dryrun was active, none of the operations above were actually executed!