QEMU support for VM tasks.

[Ferroin]

Describe the problem Support for only VirtualBox for VM tasks makes it extremely difficult for people who need to use different virtualization tools to support projects which require use of a VM to run tasks.

Describe the solution you'd like QEMU is one of the most widely deployed type-2 hypervisors around. Support for it would go a long way to enabling users who can't (or don't want to) have VirtualBox on their systems to still contribute to projects which require VM usage to run tasks.

Additional context QEMU itself is easy to use in an automated/scripted manner, and provides (limited) native VDI support. It can also run without need for hardware-assisted virtualization (albeit slowly), and doesn't need the host CPU to match the emulated CPU (also slow, but better than nothing).

Documentation is, unfortunately, somewhat lacking, but I've got some experience dealing with QEMU and am more than happy to help by answering any questions that arise. I've also got a couple of systems I can use to help with testing of any implementation.

[AenBleidd]

As far as I know (correct me if I'm wrong) QEMU can't run VirtualBox images. So you just can't switch from VirtualBox to QEMU (or any other VM hypervisor). There should be a support from project for such kind of hypervisors. And also there could be an issue that user must choose what projects to participate to, that one who support VirtualBox (several one) or QEMU (unknown at this moment how much interest will be) because it is not recommended to run different hypervisors on the same hardware simultaneously.

[Ferroin]

As far as I know (correct me if I'm wrong) QEMU can't run VirtualBox images.

It can't run them directly, but the configuration is not particularly difficult to translate (I could probably put something together in Python to do it to the required degree in a few hundred lines of code). This shouldn't matter for the case of boinc2docker, and it shouldn't be hard to deal with for other cases unless they're explicitly depending on things like USB 3.0 or multi-display support. The only potentially tricky parts are the disk images (QEMU supports VDI natively, but doesn't support 100% of the features that VBox does), and any usage of shared folders (but there are multiple ways around that provided the exact semantics of how VBox shares folders aren't required).

And also there could be an issue that user must choose what projects to participate to, that one who support VirtualBox (several one) or QEMU (unknown at this moment how much interest will be) because it is not recommended to run different hypervisors on the same hardware simultaneously.

I think you mean 'doesn't work', not 'not recommended'. With some very limited exceptions, you can't share virtualization capabilities between multiple hypervisors in the same context, which is actually the whole reason I opened this issue in the first place (I can't run VBox on most of my systems because I need QEMU on them with hardware virtualization enabled, or they require other third-party kernel modules that don't play nice with the VBox host modules).

However, QEMU can run without needing hardware virtualization support, including supporting 64-bit guests (unlike VBox, which needs hardware virtualization to support 64-bit guests), and it's generally still reasonably fast as long as your CPU architecture matches what the VM is providing.

[davidpanderson]

BOINC could support both QEMU and VBox. We'd need to

• have the client detect and report QEMU (I'll add this if someone can tell me how to detect it)
• develop a QEMU counterpart to vboxwrapper

Projects would then need to make QEMU app versions.

This is all worth doing only if a significant number of hosts have QEMU.

[adamradocz]

I don't like the idea supporting VM computing on not hardware accelerated CPUs. It's a huge waste of computational power. I think this scenario shouldn't be allowed.

[adamradocz]

What's the difference between this and Hyper-V? The Hyper-V issue was closed. #1974

[AenBleidd]

Hyper-V is Windows only

[Ferroin]

@davidpanderson

have the client detect and report QEMU (I'll add this if someone can tell me how to detect it)

Check $PATH for qemu-system-x86_64 or qemu-system-x86 depending on whether you need 64 or 32 bit support, and verify that Xen is not present (Xen uses QEMU for hardware emulation, but you can't easily run regular QEMU VM's on a Xen host), which largely amounts to detecting whether or not a daemon called xenstored is running (that will tell you not only whether Xen is present, but whether it's running or not too).

develop a QEMU counterpart to vboxwrapper

I can help with this, but don't have anywhere near enough background to do it myself. QEMU itself can easily provide a per-VM control socket (though they use the term 'monitor' for this), which is probably the interface that should be used for this (it can do most of the same types of things that VBoxManage can).

This is all worth doing only if a significant number of hosts have QEMU.

Most Windows hosts won't have it, but a non-negligible percentage of Linux and BSD hosts probably will. VBox is popular among developers and end-users, but QEMU is decidedly more popular among power users on Linux because it's a lot more flexible.

[Ferroin]

Correction, the check for the absence of Xen may not be needed, I did some more digging and it looks like it shouldn't be an issue to run with Xen as a back-end instead of KVM (and QEMU supports others too, such as Intel HAXM, macOS HyperVisor Framework, and the Windows Hypervisor Platform), other than requiring proper testing of course.

[davidpanderson]

It wouldn't hurt to have the client check for as many VM-related systems as possible, if only to get an idea of what's out there. Can someone make a matrix of what to check for and how to check for it on each of (Win, Mac, Linux)?

BTW, people have had the idea of a single wrapper that works with multiple VM systems: https://eden.dei.uc.pt/~filipius/arquivo/2011/pcgrid2011v2.pdf

[Ferroin]

libvirt may be of interest both for learning how to detect stuff, and simplifying writing a wrapper. It provides support for most of the big name hypervisors (Hyper-V, Xen, QEMU, VBox, PowerVM, VMWare, Virtuozzo, and BHyve), so I would expect that they have code either in the build scripts or the runtime to detect the presence of various hypervisors.

[lfield]

+1 for libvirt.

[Germano0]

I am running a nested virtualization with VirtualBox running inside a qemu/kvm/libvirt virtual machine

[serval2412]

Virtualbox isn't in Debian testing main repos and considering https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794466, it seems it won't be in following Debian versions too (derivatives like Ubuntu too?). Of course, it seems possible to download Virtualbox from Oracle but wouldn't like to play with dependencies.