search

nlewo / nix2container

An archive-less dockerTools.buildImage implementation
Apache License 2.0
501 stars 45 forks source link

Assertion `info.references.size() == 0' failure pulling layers containing store paths #114

Open mikepurvis opened 7 months ago

mikepurvis commented 7 months ago

This is an odd one that shouldn't be an issue for "normal" use cases, but would be nice to have fail more gracefully in exceptional ones. Basically, I'm using pullImageFromManifest (contributed in https://github.com/nlewo/nix2container/pull/77) to pull what is supposed to be an Ubuntu underlay image and then layer some Nix stuff on top of it. Unfortunately due to a misunderstanding, some Nix stuff also ended up in the underlay, including the exact curl store path that was doing the image pulling. On Hydra, we had an incomplete error message and just got error: unexpected end-of-file, but a local reproduction showed the assertion failure:

$ nix build clearpath/2.32.0-20240207044948-0#containers.testrunner
[1/0/8 built, 16/119/972 copied (389.9/7105.4 MiB), 215.7/4018.4 MiB DL] building d3a6a65979ecfb81dbc4d1fb0df93d310bca761e34b38529ab3edf011ccab91e: 100 51.6M  100 5 nix: src/libstore/store-api.cc:190: nix::StorePath nix::Store::makeFixedOutputPath(std::string_view, const nix::FixedOutputInfo&) const: Assertion `info.references.size() == 0' failed.
Aborted

Pulling the tarball and inserting it into the store was no problem:

$ nix-store --add-fixed sha256 d3a6a65979ecfb81dbc4d1fb0df93d310bca761e34b38529ab3edf011ccab91e
/nix/store/2glx74vr9q8ln8vbd512dmyqv42aa6ng-d3a6a65979ecfb81dbc4d1fb0df93d310bca761e34b38529ab3edf011ccab91e

But once I unpacked that layer archive and had a look, it was clear that both it and the build trying to produce it were using the same /nix/store/m072pj8xfk9hvv9zj6z982xqa6rgybw3-curl-8.1.1 path.

To be clear, this is almost certainly not desirable— pullImage and pullImageFromManifest should be used to pull foreign blobs into the container, not stuff containing more Nix. However, if there's a mechanism that would enable marking off those tools used (skopeo, curl, jq, etc) as not being eligible to be references, that would be great; alternatively if we don't like that, we could also inspect the layer tarballs themselves and reject them outright if a /nix/store directory is present.