Can't get the utils you provide working due to a socket_vmnet error.

[StoicDeveloper]

Hello! Thank you so much for publishing this repo. Although it seems your intention is to just provide some pointers in the right direction, I hope you don't mind if I ask for some help anyway. I'm attempting to use your mkVm and other utilities to create a nix VM on my MacBook, for running homelab stuff. I also tried this other guide but the VMs that result require additional configuration, more than I'm capable of at this point, before they could interact with the rest of the computers in my network (k3s, media servers and so on). But when I run the mkVm util from this repo, it builds the VM, but halts with this error:

$ nix run .
Failed to connect to "/var/run/socket_vmnet": No such file or directory

This occurs even with the minimal example you provided. Would you be willing to provide any hints for how to proceed?

[StoicDeveloper]

The example code was identical to what is in the repo (except for a fixed typo). But here are the relevant parts of my flake.nix:

{
  description = "Example Darwin system flake";
  # Some docs:
  # https://github.com/Gabriella439/macos-builder
  # https://guthub.com/mrkuz/macos-config
  # https://nixos.org/manual/nixpkgs/stable/#sec-darwin-builder-example-flake
  # https://nixcademy.com/posts/macos-linux-builder
  # https://nixcademy.com/posts/nix-on-macos
  # https://guthub.com/LnL7/nix-darwin
  # https://daiderd.com/nix-darwin/manual/index.html

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    nix-darwin.url = "github:LnL7/nix-darwin";
    nix-darwin.inputs.nixpkgs.follows = "nixpkgs";
    mrkuz.url = "github:mrkuz/macos-config";
  };

  outputs = inputs@{ self, mrkuz, nix-darwin, nixpkgs }:
    let
      name = "<myName>";
      system = "aarch64-darwin";
      configuration = { pkgs, ... }: {
        # List packages installed in system profile. To search by name, run:
        # $ nix-env -qaP | grep wget
        environment.systemPackages = [
          pkgs.vim
          pkgs.lunarvim
          pkgs.alacritty
          pkgs.tmux
          pkgs.git
          pkgs.ranger
          pkgs.nixfmt-classic
        ];

        # launchd.daemons.socket-vmnet = {
        #   serviceConfig = {
        #     ProgramArguments = [
        #       "/bin/sh"
        #       "-c"
        #       "/bin/wait4path ${pkgs.macos.vmnet} &amp;&amp; ${pkgs.macos.vmnet}/bin/socket_vmnet --vmnet-gateway=192.168.105.1 --vmnet-dhcp-end=192.168.105.100 /var/run/socket_vmnet"
        #       # "/bin/wait4path ${pkgs.macos.socket_vmnet} &amp;&amp; ${pkgs.macos.socket_vmnet}/bin/socket_vmnet --vmnet-gateway=${cfg.gateway} --vmnet-dhcp-end=${cfg.dhcpEnd} /var/run/socket_vmnet"
        #     ];
        #     RunAtLoad = true;
        #     StandardOutPath = "/var/log/socket-vmnet/stdout";
        #     StandardErrorPath = "/var/log/socket-vmnet/stderr";
        #     UserName = "root";
        #   };
        # };

        system.defaults.dock.persistent-apps =
          [ "/Applications/Nix Apps/Alacritty.app" ];

        nix = {
          linux-builder = {
            enable = true;
            protocol = "ssh";
            maxJobs = 4;
            config = {
              services.openssh.enable = true;
              # services.lidarr = {
              #   enable = true;
              #   openFirewall = true;
              #   dataDir = "/mnt/media/config/lidarr/";
              # };
              # fileSystems."/mnt/media" = {
              #   device = "192.168.100.154:/mnt/BigStorage/media";
              #   fsType = "nfs";
              # };
              # networking.extraHosts = ''
              #   192.168.100.180 lidarr
              #   192.168.100.180 jackett
              #   192.168.100.241 transmission
              #   192.168.100.241 sonarr
              # '';
              virtualisation = {
                darwin-builder = {
                  diskSize = 40 * 1024;
                  memorySize = 4 * 1024;
                };
                cores = 4;
              };
            };
          };
          settings.trusted-users = [ "@admin" "xian" ];
          settings.extra-trusted-users = [ "@admin" "xian" ];
        };

        # Auto upgrade nix package and the daemon service.
        services.nix-daemon.enable = true;
        # nix.package = pkgs.nix;

        # Necessary for using flakes on this system.
        nix.settings.experimental-features = "nix-command flakes";

        # Create /etc/zshrc that loads the nix-darwin environment.
        programs.zsh.enable = true; # default shell on catalina
        # programs.fish.enable = true;

        # Set Git commit hash for darwin-version.
        system.configurationRevision = self.rev or self.dirtyRev or null;

        # Used for backwards compatibility, please read the changelog before changing.
        # $ darwin-rebuild changelog
        system.stateVersion = 4;

        # The platform the configuration will be used on.
        nixpkgs.hostPlatform = "aarch64-darwin";
      };

      vm = mrkuz.utils.mkVm {
        # inherit name;
        name = "firefox";
        selfReference = self;
        targetSystem = "aarch64-linux";
        # configuration = {
        #   imports = [
        #     ./configuration.nix
        #   ];
        # };
        configuration.modules = {
          nix.enable = true;
          minimize.enable = false;
          qemuGuest = {
            autoLogin = true;
            dhcp = true;
            user = "user";
            socketVmnet = true;
          };
        };
      };
    in {
      darwinConfigurations."${name}" =
        nix-darwin.lib.darwinSystem { modules = [ configuration ]; };
      nixosModules.base = { pkgs, ... }: {
        system.stateVersion = "24.05";

        # Configure networking
        networking.useDHCP = true;
        # networking.interfaces.eth0.useDHCP = true;

        services.getty.autologinUser = "<myName>";
        users.users.<myName>.isNormalUser = true;

        # Enable passwordless ‘sudo’ for the "test" user
        users.users.<myName>.extraGroups = [ "wheel" ];
        security.sudo.wheelNeedsPassword = false;
        # services.radarr = {
        #   enable = true;
        #   openFirewall = true;
        #   dataDir = "/mnt/media/config/radarr/"; 
        # };
        environment.systemPackages = [ pkgs.nfs-utils ]; # necessary, but insufficient
        # Doesn't work, nfs server rejects all connections from this VM, even when configured insecurely
        fileSystems."/mnt/media" = {
          device = "192.168.100.154:/mnt/BigStorage/media";
          fsType = "nfs";
        };
        services.openssh.enable = true;

      };
      nixosModules.vm = { ... }: {
        # Make VM output to the terminal instead of a separate window
        virtualisation.vmVariant.virtualisation.graphics = false;
      };
      nixosConfigurations.vm = vm;
      # this is from the other guide I linked
      nixosConfigurations.darwinVM = nixpkgs.lib.nixosSystem {
        system = "aarch64-linux";
        modules = [
          self.nixosModules.base
          self.nixosModules.vm
          {
            virtualisation.vmVariant.virtualisation.host.pkgs =
              nixpkgs.legacyPackages.aarch64-darwin;
          }
        ];
      };
      packages = {
        aarch64-darwin = {
          vm = self.nixosConfigurations.vm.config.system.build.startVm;
          darwinVM = self.nixosConfigurations.darwinVM.config.system.build.vm;
        };
      };
    };
}

[StoicDeveloper]

So I managed to resolve this using the following measures: Add pkgs = mrkuz.utils.mkPkg {} to the top-level of the let clause in the flake output, which leads to: add {nixpkgs.pkgs = pkgs;} to the modules list in darwinConfigurations, and then concatenate to that list with ++ mrkuz.utils.attrsToValues mrkuz.darwinModules. It then becomes possible to use mrkuz.utils.mkVm to run a VM.

I'm still fiddling with the network settings, and NFS is unavailable inside the VM, and servers running inside are accessible from the host, but not from other computers, but this is progress.

Thanks again for providing these tools!

[mrkuz]

Happy you got it up and running! Faster than I was able to respond :)

[StoicDeveloper]

This is more tangential to the purpose of this repo, but I see that there is a k3s host that you set up, presumable it works for you. I'm aiming to do the same thing, but the VM agent can't connect to the server, even with an identical configuration to the other agents. I suspect this is for the same reason that NFS doesn't work: there is no bridge (or NAT) set up to connect the host-guest network with the rest of the local network. It seems that nix-darwin has built-in utils to do this using Wireguard, but since I assume you've solved this exact same problem already, you wouldn't happen to have some configs lying around that address this issue eh? If not then no worries, you've helped a lot already. I'm pretty new to nix and networking, but I'm sure I'll figure it out eventually.

[StoicDeveloper]

Nevermind, I managed to fix the networking issue as well, so that ping (or nfs, or ssh, or whatever) will work with the VMs. It required a relatively small change to qemu-vm.nix, but a lot of reading about networking and qemu. Here is the fork: https://gitlab.com/CreativeStoic/macos-config/

[mrkuz]

Thanks for sharing. Until now I only needed to access the VMs from the host. But great to have the solution already at hand if requirements change.