search

abbbi / virtnbdbackup

Backup utility for Libvirt / qemu / kvm supporting incremental and differential backups + instant recovery (agentless).
http://libvirtbackup.grinser.de/
GNU General Public License v3.0
330 stars 46 forks source link

virtnbdrestore fails with error: "IndexError: list index out of range" #110

Closed sor-g closed 1 year ago

sor-g commented 1 year ago

Version used 1.9.16

Describe the bug Several VMs backups were created while VMs were off. All backups were succsessful. When trying to restore (on another machine), for one of the VMs the process failed with error: "IndexError: list index out of range" Operation was redone three times - backup is successful, but restore for this particular VM fails.

Expected behavior virtnbdrestore should finish without error

Hypervisor information:

Logfiles: virtnbdrestore.log:

[2023-03-01 22:03:39] INFO lib common - printVersion [MainThread]: Version: 1.9.16 Arguments: /usr/bin/virtnbdrestore -cD -i /home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA -o /opt/VM-disk-images/restored-from-ZETASM [2023-03-01 22:03:39] INFO root virtnbdrestore - main [MainThread]: Using latest config file: [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/vmconfig.copy.xml] [2023-03-01 22:03:39] INFO virt client - getDomainDisks [MainThread]: Skipping attached [cdrom] device: [sda]. [2023-03-01 22:03:39] INFO root virtnbdrestore - createDiskFile [MainThread]: Create virtual disk [/opt/VM-disk-images/restored-from-ZETASM/test2.qcow2] format: [qcow2] size: [21474836480] based on: [virtnbdbackup.0] [2023-03-01 22:03:39] INFO root virtnbdrestore - getQcowConfig [MainThread]: Using QCOW options from backup file: [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/sdb.copy.qcow.json] [2023-03-01 22:03:39] INFO root virtnbdrestore - startNbd [MainThread]: Starting local NBD server on socket: [/var/tmp/virtnbdbackup.139586] [2023-03-01 22:03:39] INFO nbd client - _printVersion [MainThread]: libnbd version: 1.6.0 [2023-03-01 22:03:39] INFO root virtnbdrestore - startNbd [MainThread]: Started NBD server, PID: [139590] [2023-03-01 22:03:39] INFO nbd client - connect [MainThread]: Waiting until NBD server at [nbd+unix:///sdb?socket=/var/tmp/virtnbdbackup.139586] is up. [2023-03-01 22:03:40] INFO nbd client - _getBlockInfo [MainThread]: Using Maximum Block size supported by NBD server: [33554432] [2023-03-01 22:03:40] INFO nbd client - connect [MainThread]: Connection to NBD backend succeeded. [2023-03-01 22:03:40] INFO root virtnbdrestore - restoreData [MainThread]: Applying data from backup file [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/sdb.copy.data] to target file [/opt/VM-disk-images/restored-from-ZETASM/test2.qcow2]. [2023-03-01 22:04:18] INFO root virtnbdrestore - restoreData [MainThread]: End of stream, [3674341376] bytes of data processed [2023-03-01 22:04:18] INFO virt client - adjustDomainConfig [MainThread]: Removing uuid setting from vm config. [2023-03-01 22:04:18] INFO virt client - adjustDomainConfig [MainThread]: Changing name from [test2] to [restore_test2]

command output on terminal:

[root@YETA ~]# virtnbdrestore -cD -i /home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA -o /opt/VM-disk-images/restored-from-ZETASM [2023-03-01 22:03:39] INFO lib common - printVersion [MainThread]: Version: 1.9.16 Arguments: /usr/bin/virtnbdrestore -cD -i /home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA -o /opt/VM-disk-images/restored-from-ZETASM [2023-03-01 22:03:39] INFO root virtnbdrestore - main [MainThread]: Using latest config file: [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/vmconfig.copy.xml] [2023-03-01 22:03:39] INFO virt client - getDomainDisks [MainThread]: Skipping attached [cdrom] device: [sda]. [2023-03-01 22:03:39] INFO root virtnbdrestore - createDiskFile [MainThread]: Create virtual disk [/opt/VM-disk-images/restored-from-ZETASM/test2.qcow2] format: [qcow2] size: [21474836480] based on: [virtnbdbackup.0] [2023-03-01 22:03:39] INFO root virtnbdrestore - getQcowConfig [MainThread]: Using QCOW options from backup file: [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/sdb.copy.qcow.json] [2023-03-01 22:03:39] INFO root virtnbdrestore - startNbd [MainThread]: Starting local NBD server on socket: [/var/tmp/virtnbdbackup.139586] [2023-03-01 22:03:39] INFO nbd client - _printVersion [MainThread]: libnbd version: 1.6.0 [2023-03-01 22:03:39] INFO root virtnbdrestore - startNbd [MainThread]: Started NBD server, PID: [139590] [2023-03-01 22:03:39] INFO nbd client - connect [MainThread]: Waiting until NBD server at [nbd+unix:///sdb?socket=/var/tmp/virtnbdbackup.139586] is up. [2023-03-01 22:03:40] INFO nbd client - _getBlockInfo [MainThread]: Using Maximum Block size supported by NBD server: [33554432] [2023-03-01 22:03:40] INFO nbd client - connect [MainThread]: Connection to NBD backend succeeded. [2023-03-01 22:03:40] INFO root virtnbdrestore - restoreData [MainThread]: Applying data from backup file [/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/sdb.copy.data] to target file [/opt/VM-disk-images/restored-from-ZETASM/test2.qcow2]. {'checkpointName': 'virtnbdbackup.0', 'compressed': False, 'compressionMethod': 'lz4', 'dataSize': 3674341376, 'date': '2023-03-01T20:36:45.310806', 'diskFormat': 'qcow2', 'diskName': 'sdb', 'incremental': False, 'parentCheckpoint': '', 'streamVersion': 2, 'virtualSize': 21474836480} [2023-03-01 22:04:18] INFO root virtnbdrestore - restoreData [MainThread]: End of stream, [3674341376] bytes of data processed [2023-03-01 22:04:18] INFO virt client - adjustDomainConfig [MainThread]: Removing uuid setting from vm config. [2023-03-01 22:04:18] INFO virt client - adjustDomainConfig [MainThread]: Changing name from [test2] to [restore_test2] Traceback (most recent call last): File "/usr/bin/virtnbdrestore", line 707, in main() File "/usr/bin/virtnbdrestore", line 693, in main restConfig = restore(args, ConfigFile, virtClient) File "/usr/bin/virtnbdrestore", line 475, in restore restConfig = virtClient.adjustDomainConfig(args, disk, vmConfig, targetFile) File "/usr/lib/python3.6/site-packages/libvirtnbdbackup/virt/client.py", line 317, in adjustDomainConfig originalFile = disk.xpath("source")[0].get("file") IndexError: list index out of range

Workaround:

abbbi commented 1 year ago

its unable to detect the original disk file to replace the pathes, i need the virtual machine configuratoin file to check where the issue is to reproduce. You can restore without -cD options and adjust the restored config manually to workaround. Guess the virtual machine config uses some kind of disk layout notation which is correctly handled during backup but not implemented during restore with adjust options.

abbbi commented 1 year ago

i need at least the disk section from the file:

/home/cccc/Downloads/VM-images-BACKs-frm-ZETASM/BACK-test2-for-migration-to-YETA/vmconfig.copy.xml

to analyse. (or the complete file if it doesnt include sensitive data)

sor-g commented 1 year ago

Dear Michael, here is the xml: vmconfig.copy.xml.zip

abbbi commented 1 year ago

problem is the ordering of the devices in the VM. The CDROM device (which his removed during adjustment of configuration) is listed in the config before the first real disk device. This commit should fix this:

https://github.com/abbbi/virtnbdbackup/commit/a6f0e42fc3c37a7a7868aef4dc63d429c2687cd7

can you test?

sor-g commented 1 year ago

Just tested. Issue is resolved in v. 1.9.17 Thanks for quick action!