abbbi / virtnbdbackup

Backup utility for Libvirt / qemu / kvm supporting incremental and differential backups + instant recovery (agentless).
http://libvirtbackup.grinser.de/
GNU General Public License v3.0
330 stars 46 forks source link

If user adds new disk, then virtnbdbackup could create new full backup for this disk. #135

Open alonezw opened 1 year ago

alonezw commented 1 year ago

Version used 1.9.42

Describe the bug If virtnbdbackup works in incremental mode with some number of disks, and user adds another one, then backup brokes with error.

Expected behavior If user adds new disk, then virtnbdbackup must create new full backup for this disk.

Hypervisor information:

Logfiles: [2023-09-12 16:14:33] INFO lib common - printVersion [MainThread]: Version: 1.9.42 Arguments: /bin/virtnbdbackup --domain backup-test --level inc --compress --output /kvm/backups/0c06fc90-c706-4342-825b-29647658b840_backup-test_1 [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Backup level: [inc] [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Compression enabled, level [2] [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Libvirt library version: [9000000] [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Backup will save [3] attached disks. [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Concurrent backup processes: [3] [2023-09-12 16:14:33] INFO root checkpoint - redefine [MainThread]: Loading checkpoint list from: [/kvm/backups/0c06fc90-c706-4342-825b-29647658b840_backup-test_1/checkpoints] [2023-09-12 16:14:33] INFO root checkpoint - create [MainThread]: Checkpoint handling. [2023-09-12 16:14:33] INFO root checkpoint - create [MainThread]: Next checkpoint id: [3]. [2023-09-12 16:14:33] INFO root checkpoint - create [MainThread]: Parent checkpoint name [virtnbdbackup.2]. [2023-09-12 16:14:33] INFO root checkpoint - create [MainThread]: Using checkpoint name: [virtnbdbackup.3]. [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Local NBD Endpoint socket: [/var/tmp/virtnbdbackup.2164493] [2023-09-12 16:14:33] INFO root virtnbdbackup - main [MainThread]: Temporary scratch file target directory: [/var/tmp] [2023-09-12 16:14:33] INFO root virtnbdbackup - startBackupJob [MainThread]: Starting backup job. [2023-09-12 16:14:33] INFO fs fs - freeze [MainThread]: Freezed [1] filesystems. [2023-09-12 16:14:33] INFO fs fs - thaw [MainThread]: Thawed [1] filesystems. [2023-09-12 16:14:33] ERROR root virtnbdbackup - startBackupJob [MainThread]: Failed to start backup: [checkpoint inconsistent: missing or broken bitmap 'virtnbdbackup.2' for disk 'vdc']

Workaround: no

abbbi commented 1 year ago

Its correctly failing because the checkpoint information about the disks is inconsistent ( the freshly added disk does not have an bitmap tied to the (prior) checkpoint)

Thats okey. There is currently no logic that attempts to check wether if an virtual machine amount of disks was altered between full and incremental backup. That would be an enhancement, as it stands. Currently the user at least notices theres something wrong.

Also, falling back to a full backup would mean backup needs to go to a new directory, as the current backup folder used already includes an full backup (complete new backup chain must be started to a fresh, empty directory), and following incremental backups must be changed to this target directory then too. Its not an easy fix and there all kinds of scenarios i could imagine this will result in confusing situations.

I dont think ill implement this, ill leave it to the users sane behavior to know a full backup should be done next if hes altering the virtual machine configuration in such ways, or script around this using pre/post scripting.

Also i dont know if the restore utility could handle this correctly without changes and the feature would become even more interesting in transient virtual machine (clustered) environments where re-defining the checkpoints is mandatory to continue backup chains.

alonezw commented 1 year ago

Ok, i undestand. Thanks. :)