pdtpartners / nix-snapshotter

Brings native understanding of Nix packages to containerd
MIT License
532 stars 15 forks source link

resolvedByNix=false leads to infinite recursion #131

Closed psionic-k closed 5 months ago

psionic-k commented 6 months ago

I probably don't understand the high level yet. I just wanted OCI containers built with sensible size and store efficiency with Nix.

I needed containers in the cloud, so I was presuming the special nix store integration is irrelevant and resolvedByNix should be false.

          image = pkgs.nix-snapshotter.buildImage {
            name = "hello";
            resolvedByNix = false;
            config.entrypoint = [ "${hello}/bin/hello" ];
          };

While resolvedByNix set to true will succeed, when set to false, the evaluation goes into infinite recursion.

warning: Git tree '/home/satoshi/Desktop/prizeforge/pocs/test-cross' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-image-hello.tar'
         whose name attribute is located at /nix/store/7x6ahdry0dp4dd7bi1lips656y727175-source/pkgs/stdenv/generic/make-derivation.nix:348:7

       … while evaluating attribute 'buildCommand' of derivation 'nix-image-hello.tar'

         at /nix/store/7x6ahdry0dp4dd7bi1lips656y727175-source/pkgs/build-support/trivial-builders/default.nix:87:14:

           86|       enableParallelBuilding = true;
           87|       inherit buildCommand name;
             |              ^
           88|       passAsFile = [ "buildCommand" ]

       error: infinite recursion encountered

       at «none»:0: (source not available)

Can drill into this if it indeed isn't a weird choice for my use case. TIA

elpdt852 commented 5 months ago

I probably don't understand the high level yet. I just wanted OCI containers built with sensible size and store efficiency with Nix.

I needed containers in the cloud, so I was presuming the special nix store integration is irrelevant and resolvedByNix should be false.

Hi @psionic-k, thanks for the report. Turns out when the tag is also not set, it defaulted to something that depended on itself, causing an infinite recursion. This has been fixed in the PR above and I’ll also make a new release of nix-snapshotter as well.

The resolvedByNix is a feature to integrate with Kubernetes, to resolve images directly with the Nix store. This removes the need to run a Docker Registry if you are only using nix-snapshotter images.

If you’re not using Kubernetes, you can it leave it as the default (false).

psionic-k commented 5 months ago

I'm using Kubernetes but where would I smartly keep a nix store when using cloud based kubernetes? Only asking in case images are sub-optimal in that case.

Thanks for patch and feedback.

elpdt852 commented 5 months ago

@psionic-k You should be able to customize the underlying nodes on EKS, and likely GKE as well. See the Create a managed node group that uses containerd section from this blog post: https://blog.realvarez.com/using-estargz-to-reduce-container-startup-time-on-amazon-eks/

It's about stargz-snapshotter but the instructions will be the same as nix-snapshotter. You'll also need to install nix, so you can try: https://github.com/DeterminateSystems/nix-installer