astro / microvm.nix

NixOS MicroVMs
https://astro.github.io/microvm.nix/
MIT License
1.25k stars 94 forks source link

virtio_mem device error #191

Open birkb opened 7 months ago

birkb commented 7 months ago

The logs of my VMs show this error, but i cannot find further infos about it:

virtio_mem virtio7: unknown error, marking device broken: -22

config

microvm.nixosModules.microvm
    {
      microvm = {
        hypervisor = "cloud-hypervisor";
        mem = 1024;
        balloonMem = 1024;
        vcpu = 2;
        interfaces = [
          {
            type = "macvtap";
            id = "vm-m1-01";
            mac = "1C:83:41:01:21:01";
            macvtap = {
              link = "enp2s0";
              mode = "private";
            };
          }
          {
            type = "macvtap";
            id = "vm-m1-02";
            mac = "1C:83:41:01:21:02";
            macvtap = {
              link = "enp3s0";
              mode = "private";
            };
          }
        ];
        volumes = [ 
          {
          mountPoint = "/";
            autoCreate = true;
            fsType = "ext4";
            label = "root";
            size = 65535;
            image = "root.img";
          }
        ];
        shares = [
          {
          source = "/nix/store";
            mountPoint = "/nix/.ro-store";
            tag = "ro-store";
            proto = "virtiofs";
          }
          {
            source = "/root";
            mountPoint = "/root";
            tag = "ro-root";
            proto = "virtiofs";
          }
        ];
      };
    }

nix-info

nix-shell -p nix-info --run "nix-info -m"

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.73, NixOS, 23.11 (Tapir), 23.11.20240118.d1b73d6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
birkb commented 7 months ago

The journal of the VM contains these entries

m2 kernel: virtio_mem virtio7: start address: 0x100000000
m2 kernel: virtio_mem virtio7: region size: 0x40000000
m2 kernel: virtio_mem virtio7: device block size: 0x200000
m2 kernel: virtio_mem virtio7: nid: 0
m2 kernel: virtio_mem virtio7: memory block size: 0x8000000
m2 kernel: virtio_mem virtio7: subblock size: 0x200000
m2 kernel: virtio_mem virtio7: plugged size: 0x0
m2 kernel: virtio_mem virtio7: requested size: 0x40000000
m2 systemd[1]: Finished Rebuild Journal Catalog.
m2 kernel: virtio_mem virtio7: unknown error, marking device broken: -22
astro commented 7 months ago

This one probably needs a qemu/linux deep-dive.

Why do you want ballooning? As a more viable alternative, split the memory-hungry but temporary task off into a temporary VM if you can.

birkb commented 7 months ago

The host has 8GB RAM and i want to run 2 VMs on it

The VMs have no swap configured and because of that balloning makes little sense in the VMs. I will let the host manage the swapping in case temporarily more than 8GB are required on the host or the VMs...

The question (for balloning use cases) still is why the module cannot be properly loaded...