fog / fog-libvirt

libvirt provider for fog
MIT License
16 stars 42 forks source link

fix disk type for block devices on QEMU >= 6.0 #116

Closed lero closed 11 months ago

lero commented 2 years ago

since QEMU version 6.0 [1], you can't use the file block driver with block devices.

[1] https://wiki.qemu.org/ChangeLog/6.0#Incompatible_changes

lero commented 2 years ago

a bit more context...

RHEL 8.6 ships QEMU 6.2 and anything that uses fog-libvirt, like https://theforeman.org, is currently broken while creating VMs. RHEL 8.5 shipped with QEMU 4.2, so it's not affected.

any other distro with QEMU >= 6.0 is also impacted.

lzap commented 2 years ago

Hey thanks, do you know what happens to older versions hypervisors? My thinking is - let's not break existing deployments.

lero commented 2 years ago

I guess we should be ok with block devices. Even on RHEL 6 they say to use "block": https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/sect-virtualization-adding_storage_devices_to_guests-adding_hard_drives_and_other_block_devices_to_a_guest

But, if we have file-based storage (like a .img), then it needs "file". So I think we will need to guess if it's a block device or a normal file before :/

rwf14f commented 2 years ago

I can confirm that using a block device definition fails with file-based storage. It lets you define the VM but qemu fails with an error when you try to start it. Each volume has a type, if you make that available in the erb then you should be able to use it in the template to determine whether to create block or file entry. The catch is that the type is set by libvirt after you create the volume, so you have to query it from the hypervisor. On our systems it looks like the volume type is 0 for <volume type='block'> and 1 for <volume type='file'>.

antaflos commented 1 year ago

This is hitting us on hypervisor hosts running Ubuntu 22.04. Foreman (Fog) is creating incompatible libvirt/qemu definitions for disks and VMs created in such a way are unable to start.

What can we do here?

lzap commented 1 year ago

I am hoping to see some flag that can be provided in the new vm method so when this is an issue for users of older hypervisor we would have a solution. But since this is Ruby a hotfix could be done as well. @ekohl ?

antaflos commented 1 year ago

Now the question is what a proper implementation is. One solution is a parameter, but it's really unclear how the volumes get set for creation. Another is that you may be able to guess it's a block device if it comes from /dev. I can't really foresee all issues, but I think it's fairly safe to assume no files are stored in /dev and no block devices on other paths.

I think that is as reasonable an assumption as you can make.

Regular files under /dev (usually?) do not survive a reboot so anything that's not a (block) device will not be permanent anyway (we experienced this after we created a VM with a backing qcow2 image that was accidentally created under /dev, and the VM would not start after a hypervisor host reboot because its disk was gone).

As for considering block devices outside of /dev: I don't think it is worth to take that possibility into account.

antaflos commented 1 year ago

Gentle bump since this is becoming a major hindrance in out virtualisation infrastructure. Is there an acceptable solution so that we should wait for the release of a new version of fog-libvirt? Or should we monkeypatch or fork and package fog-libvirt ourselves internally?

ekohl commented 11 months ago

I've opened https://github.com/fog/fog-libvirt/pull/129 which implements the idea, but I don't have the capacity to test it.

ekohl commented 11 months ago

129 has been merged.