pdtpartners / nix-snapshotter

Brings native understanding of Nix packages to containerd
MIT License
532 stars 15 forks source link

extending the PATH envvar for containerd-rootless #121

Closed msackman closed 6 months ago

msackman commented 6 months ago

I'm trying to explore whether I can get nix-snapshotter working with containerd with gvisor.

I've got the configuration tweaks done - https://gvisor.dev/docs/user_guide/containerd/quick_start/

    virtualisation.containerd.rootless = {
      enable = true;
      nixSnapshotterIntegration = true;
      settings = {
        plugins."io.containerd.runtime.v1.linux" = {
          shim_debug = true;
        };
        plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc = {
          runtime_type = "io.containerd.runc.v2";
        };
        plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc = {
          runtime_type = "io.containerd.runsc.v1";
        };
      };
    };

The current challenge I'm facing is that I need gvisor's runsc and containerd-shim-runsc-v1 binaries to be in the path of containerd, ... and having looked through https://github.com/pdtpartners/nix-snapshotter/blob/main/modules/common/containerd-rootless.nix I reckon it's actually the containerd-rootless package where I need to tweak the path.

Normally, I would expect to be able to do something like this:

let {
  containerd = pkgs.containerd.overrideAttrs (oldAttrs: {
    extraPath = lib.makeBinPath [ pkgs.gvisor ];
  });
} in {
    virtualisation.containerd.rootless = {
      enable = true;
      nixSnapshotterIntegration = true;
      package = containerd;
    }
}

but that's not working - it seems to me that containerd-rootless isn't a package that I can tweak in this way.

I've spent a bit of time searching the docs for containerd to see if there's some way I can provide paths in the config files, but I can't see anything there, which is why I've been attempting to tweak the PATH env var instead.

Do you have any suggestions, or am I so far off the beaten track now that this is a "fork and edit it" job?

msackman commented 6 months ago

Brilliant, thank you!