Closed lero closed 11 months ago
a bit more context...
RHEL 8.6 ships QEMU 6.2 and anything that uses fog-libvirt, like https://theforeman.org, is currently broken while creating VMs. RHEL 8.5 shipped with QEMU 4.2, so it's not affected.
any other distro with QEMU >= 6.0 is also impacted.
Hey thanks, do you know what happens to older versions hypervisors? My thinking is - let's not break existing deployments.
I guess we should be ok with block devices. Even on RHEL 6 they say to use "block": https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/sect-virtualization-adding_storage_devices_to_guests-adding_hard_drives_and_other_block_devices_to_a_guest
But, if we have file-based storage (like a .img), then it needs "file". So I think we will need to guess if it's a block device or a normal file before :/
I can confirm that using a block device definition fails with file-based storage. It lets you define the VM but qemu fails with an error when you try to start it.
Each volume has a type, if you make that available in the erb then you should be able to use it in the template to determine whether to create block or file entry. The catch is that the type is set by libvirt after you create the volume, so you have to query it from the hypervisor. On our systems it looks like the volume type is 0 for <volume type='block'>
and 1 for <volume type='file'>
.
This is hitting us on hypervisor hosts running Ubuntu 22.04. Foreman (Fog) is creating incompatible libvirt/qemu definitions for disks and VMs created in such a way are unable to start.
What can we do here?
I am hoping to see some flag that can be provided in the new vm method so when this is an issue for users of older hypervisor we would have a solution. But since this is Ruby a hotfix could be done as well. @ekohl ?
Now the question is what a proper implementation is. One solution is a parameter, but it's really unclear how the volumes get set for creation. Another is that you may be able to guess it's a block device if it comes from /dev. I can't really foresee all issues, but I think it's fairly safe to assume no files are stored in /dev and no block devices on other paths.
I think that is as reasonable an assumption as you can make.
Regular files under /dev
(usually?) do not survive a reboot so anything that's not a (block) device will not be permanent anyway (we experienced this after we created a VM with a backing qcow2 image that was accidentally created under /dev
, and the VM would not start after a hypervisor host reboot because its disk was gone).
As for considering block devices outside of /dev
: I don't think it is worth to take that possibility into account.
Gentle bump since this is becoming a major hindrance in out virtualisation infrastructure. Is there an acceptable solution so that we should wait for the release of a new version of fog-libvirt? Or should we monkeypatch or fork and package fog-libvirt ourselves internally?
I've opened https://github.com/fog/fog-libvirt/pull/129 which implements the idea, but I don't have the capacity to test it.
since QEMU version 6.0 [1], you can't use the file block driver with block devices.
[1] https://wiki.qemu.org/ChangeLog/6.0#Incompatible_changes