search

lima-vm / lima

Linux virtual machines, with a focus on running containers
https://lima-vm.io/
Apache License 2.0
15.04k stars 589 forks source link

Converting qcow2 images to raw is too slow #2579

Open nirs opened 2 weeks ago

nirs commented 2 weeks ago

Description

Based on the logs, converting ubuntu server cloud image (xxx MiB) to raw format takes 17 seconds. The same operation using qemu-img convert takes 1.8 seconds.

Example log:

time="2024-09-02T01:43:25+03:00" level=info msg="Converting \"/Users/nsoffer/.lima/cluster/basedisk\" (qcow2) to a raw disk \"/Users/nsoffer/.lima/cluster/diffdisk\""
...
time="2024-09-02T01:43:42+03:00" level=info msg="Expanding to 20GiB"

Same with qemu-img

% time qemu-img convert -f qcow2 -O raw ~/.lima/cluster/basedisk diffdisk
qemu-img convert -f qcow2 -O raw ~/.lima/cluster/basedisk diffdisk  2.37s user 1.90s system 241% cpu 1.768 total

Lima has nice progress bar during the slow convert, but qemu-img is fast enough so no progress bar is needed. It has also a progress bar option that can be used to extract progress if needed.

Fix:

jandubois commented 2 weeks ago

Fix:

  • use qemu-img convert if available

It would be better to fix the speed of the builtin conversion so it will be fast even when QEMU is not installed. Given that the default emulation in Lima 1.0 will be VZ, qemu will be an optional dependency.

nirs commented 2 weeks ago

I don't think that reinventing qemu-img good direction. The time spent on it can be spent on features that that add values to users. qemu-img is efficient, supports all images formats, well maintained, and available everywhere.

afbjorklund commented 2 weeks ago

You can default to qemu-img (where available), and then fallback to the library as a slower fallback option?

We have used this trick elsewhere, like with SFTP or with XZ. The downside is having two code paths to test...

AkihiroSuda commented 2 weeks ago

qemu-img is efficient, supports all images formats, well maintained, and available everywhere.

On macOS, it is hard to install qemu-img when Homebrew/MacPorts/nix is disallowed due to employers' policy

AkihiroSuda commented 2 weeks ago

This may have a room for optimization https://github.com/lima-vm/go-qcow2reader/blob/v0.1.2/image/qcow2/qcow2.go#L795-L800