Operation not supported: creating checkpoint for incremental backup is not supported yet

[rdziwinski]

Version used 2.0

Describe the bug Hello, I wanna create a live incremental/full backup and I get the error as below. I'm trying to search for why it's unsupported but I can't find any information. My image:

image: /var/lib/one/datastores/167/814/disk.0
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 442 MiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    bitmaps:
        [0]:
            flags:
                [0]: auto
            name: one-814-0
            granularity: 65536
    refcount bits: 16
    corrupt: false
    extended l2: false

Hypervisor information:

• OS: Debian 11.8
• HV type: OpenNebula
• Libvirtd 8.0.0
• QEMU 5.2.0

Logfiles:

root@node10:/tmp/one814# virtnbdbackup -v -d one-814 -l auto -o /tmp/one814 [2024-03-13 11:08:20] INFO lib common - printVersion [main]: Version: 2.0 Arguments: /usr/bin/virtnbdbackup -v -d one-814 -l auto -o /tmp/one814 [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Backup level: [auto] [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Backup mode auto, target folder is empty: executing full backup. [2024-03-13 11:08:20] DEBUG output target - create [main]: Create: /tmp/one814/checkpoints [2024-03-13 11:08:20] DEBUG virt client - _connect [main]: Libvirt URI: [qemu:///system] [2024-03-13 11:08:20] DEBUG virt client - _connect [main]: Hostname: [node10.nebula.infra.k.pl] [2024-03-13 11:08:20] DEBUG virt client - _connect [main]: Connect libvirt using open function. [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Libvirt library version: [8000000] [2024-03-13 11:08:20] DEBUG virt client - getDomainDisks [main]: Disk [vda]: file notation [2024-03-13 11:08:20] INFO root disktype - Optical [main]: Skipping attached [cdrom] device: [hda]. [2024-03-13 11:08:20] DEBUG virt client - getDomainDisks [main]: Device list: [DomainDisk(target='vda', format='qcow2', filename='disk.0', path='/var/lib/one//datastores/167/814/disk.0', backingstores=[])] [2024-03-13 11:08:20] DEBUG virt client - getDomainInfo [main]: No setting [loader] found: 'NoneType' object has no attribute 'text' [2024-03-13 11:08:20] DEBUG virt client - getDomainInfo [main]: No setting [nvram] found: 'NoneType' object has no attribute 'text' [2024-03-13 11:08:20] DEBUG virt client - getDomainInfo [main]: No setting [kernel] found: 'NoneType' object has no attribute 'text' [2024-03-13 11:08:20] DEBUG virt client - getDomainInfo [main]: No setting [initrd] found: 'NoneType' object has no attribute 'text' [2024-03-13 11:08:20] DEBUG virt client - getDomainInfo [main]: Domain Info: [{}] [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Backup will save [1] attached disks. [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Concurrent backup processes: [1] [2024-03-13 11:08:20] INFO root checkpoint - create [main]: Loading checkpoints from: [/tmp/one814/one-814.cpt] [2024-03-13 11:08:20] INFO root checkpoint - redefine [main]: Loading checkpoint list from: [/tmp/one814/checkpoints] [2024-03-13 11:08:20] INFO root checkpoint - create [main]: Checkpoint handling. [2024-03-13 11:08:20] DEBUG root checkpoint - removeAll [main]: Cleaning up persistent storage /tmp/one814/checkpoints [2024-03-13 11:08:20] INFO root checkpoint - create [main]: Using checkpoint name: [virtnbdbackup.0]. [2024-03-13 11:08:20] DEBUG root checkpoint - create [main]: Checkpoint info: [{'name': 'virtnbdbackup.0', 'parent': '', 'file': '/tmp/one814/one-814.cpt'}]. [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Local NBD Endpoint socket: [/var/tmp/virtnbdbackup.2201523] [2024-03-13 11:08:20] INFO root virtnbdbackup - main [main]: Temporary scratch file target directory: [/var/tmp] [2024-03-13 11:08:20] DEBUG output target - create [main]: Create: /var/tmp [2024-03-13 11:08:20] INFO root job - start [main]: Starting backup job. [2024-03-13 11:08:20] DEBUG virt client - _createBackupXml [main]: Using scratch file: /var/tmp/backup.9FRK1.vda [2024-03-13 11:08:20] DEBUG root xml - indent [main]:

[2024-03-13 11:08:20] DEBUG root xml - indent [main]:

Backup checkpoint virtnbdbackup.0

[2024-03-13 11:08:20] DEBUG fs fs - freeze [main]: Attempting to freeze filesystems. [2024-03-13 11:08:20] INFO fs fs - freeze [main]: Freezed [1] filesystems. [2024-03-13 11:08:20] DEBUG virt client - startBackup [main]: Starting backup job via libvirt API. [2024-03-13 11:08:20] DEBUG fs fs - thaw [main]: Attempting to thaw filesystems. [2024-03-13 11:08:20] INFO fs fs - thaw [main]: Thawed [1] filesystems. [2024-03-13 11:08:20] ERROR root job - start [main]: Failed to start backup: [Operation not supported: creating checkpoint for incremental backup is not supported yet]

[abbbi]

the libvirt version used should support this, but the error message comes from libvirt directly, not from virtnbdbackup. I dont know if opennebula actually patch their libvirt and disable features?

save checkpoint to file test.xml

<domaincheckpoint>
  <description>Backup checkpoint</description>
  <name>virtnbdbackup.0</name>
  <disks>
    <disk name="vda"/>
  </disks>
</domaincheckpoint>

try with:

virsh checkpoint-create --xmlfile test.xml one-814

if it prints same error, please reach out to opennebula guys.

[abbbi]

i think your qemu version might be too old or does not have the capabilities. from libvirt qemu_backup.c:

    if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_INCREMENTAL_BACKUP)) {
        size_t i;

        if (chkdef) {
            virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s",
                           _("creating checkpoint for incremental backup is not supported yet"));                                                                                                                                                                                
            goto endjob;
        }    

maybe it helps to enable capability via libvirt as described in:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--760-debian-bullseye-ubuntu-20x

otherwise your qemu version is too old.

[abbbi]

it seems all actual opennebula versions available (up to 6.8) are based on debian bullseye. No packages for Debian bookworm are available yet. While the libvirt/qemu versions in debian bulleye do support the incremental backup feature, it is not enabled by default.

See following from the README:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--760-debian-bullseye-ubuntu-20x

you could try to enable the capability on opennebula libvirt using virsh too, i however dont know if opennebula will keep this setting permanently if you change the libvirt xml using virsh.

[rdziwinski]

Hello, thank you a lot for your support. Adding capabilities allows for create of checkpoint.