search

abbbi / virtnbdbackup

Backup utility for Libvirt / qemu / kvm supporting incremental and differential backups + instant recovery (agentless).
http://libvirtbackup.grinser.de/
GNU General Public License v3.0
330 stars 46 forks source link

Progress show TB of data during backup of ~61GB disk #160

Closed slu-010101 closed 6 months ago

slu-010101 commented 6 months ago

Version used virtnbdbackup --version 2.0

Describe the bug Progress show TB of data during backup of ~61GB disk, this problem is reproducible.

Hypervisor information: Debian 12.5 libvirt-daemon 9.0.0-4 virtnbdbackup.log Screenshot 2024-03-19_10-17

abbbi commented 6 months ago

hm.. seems that TQDM (https://github.com/tqdm/tqdm) measuring is off here.

The Logfile reports ~60GB data size to backup:

root disk - backup [sda]: 64424509440 bytes of data extents to backup

but the progress bar scales up to TB.. Progressbar is setup with Bytes as unit but with unit_scale option:

https://github.com/abbbi/virtnbdbackup/blob/master/libvirtnbdbackup/common.py#L226

Maybe the parameter for unit_scale=True is wrong and needs an actual size:

unit_scale: bool or int or float, optional
If 1 or True, the number of iterations will be reduced/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc.) [default: False]. If any other non-zero number, will scale total and n.

the Performance with 357 GB/s is pretty tight too. Maybe the Backup finishes so fast TQDM is unable to calcuclate / measure correctly?

abbbi commented 6 months ago

while i cannot reproduce your situation, the actual progressbar throughput and progress was off since change for issue #133, this has been fixed with following commit:

https://github.com/abbbi/virtnbdbackup/commit/3451ed1a91b4f4ea3e643d7662540a765680bdac

im not sure if it actually solves this problem, but may be related. You could try version 2.1 packages from the artifacts as soon as built has finished, which will include recent changes:

https://github.com/abbbi/virtnbdbackup/actions/runs/8341161475

slu-010101 commented 6 months ago

Here the screenshot with 2.1. 2 1_2024-03-19_13-26

slu-010101 commented 6 months ago

...and the progress bar is now complete different. In 2.0 it finished after seconds and in 2.1 its slower and equals to the disk size and ~900MB/s write performance.

abbbi commented 6 months ago

that looks OK for me .. The complete progress bar was offset in version 2.0 and probably prior because wrong values were used to update it.