search

astro / microvm.nix

NixOS MicroVMs
https://astro.github.io/microvm.nix/
MIT License
1.24k stars 93 forks source link

cloud-hypervisor: allow using a capability wrapper #255

Open SomeoneSerge opened 2 months ago

SomeoneSerge commented 2 months ago

I'd like to be able to have cloud-hypervisor create TAP devices without sudo. I'm still testing this, and I'm not sure how best to handle the graphics/non-graphics variants and versions, but the basic functionality of trying a "system" cloud-hypervisor and falling back to the store path is easy to achieve

astro commented 2 months ago

These are quite a number of changes to give more permissions to cloud-hypervisor. I wonder about alternatives.

Apart from tuntap the nixosModules.host systemd units also start virtiofsd. Maybe we could detect whether microvm-run was started through systemd, and if it was not, start virtiofsd instances and create the tap interface in a privileged tool script that has small concise functionality.

SomeoneSerge commented 2 months ago

These are quite a number of changes to give more permissions to cloud-hypervisor

Does feel a bit ad hoc, although conceptually I feel like "a fallback-capable wrapper that tries to use a system tool" should not be an uncommon scenario

Maybe we could detect whether microvm-run was started through systemd

Tbh, I've been using this with nix run .#...declaredRunner:)

SomeoneSerge commented 1 month ago

Oh, I just noticed I get an error with qemu as well: microvm@someone-0tier: -netdev tap,id=p-uvm-0tier,ifname=p-uvm-0tier,script=no,downscript=no: could not configure /dev/net/tun (p-uvm-0tier): Operation not permitted (I do have the qemu-bridge-helper deployed, EDIT: but I guess it's only used for type = "bridge"; I was only going to create TAPs)

So what is the general story for dynamic/imperative microVMs and security wrappers right now? A tangential question, getting far outside the scope of the PR, is what's the situation with managing virtiofsd services for imperative microVMs. I see there's an option to deploy a systemd template service for running these via supervisord, but that seems like a microvmCommand-specific solution...

SomeoneSerge commented 1 month ago

I feel like "a fallback-capable wrapper that tries to use a system tool" should not be an uncommon scenario

Would you be more comfortable merging this if I tried to push a more generic wrapper implementation into Nixpkgs? 🙃