nlewo / nix2container

An archive-less dockerTools.buildImage implementation
Apache License 2.0
501 stars 45 forks source link

devenv: no image using skopeo copy from Nix source to docker-daemon target #112

Closed mcdonc closed 7 months ago

mcdonc commented 7 months ago

Howdy!

I submitted https://github.com/containers/skopeo/issues/2204 as a result of using devenv to try to generate and push an image into the local docker-daemon registry. (EDIT: since then, I closed it, because it appears that this is not a skopeo bug).

The details are all present in that issue, but basically the skopeo copy command that is meant to push an image into the docker daemon appears to succeed, but no image actually winds up there.

I tried it with the skopeo present on nixos-23.11 (1.13) and the skopeo present on nixpkgs-unstable (1.14) with the same result. (EDIT: I was wrong, using 1.14.1 made the issue visible rather than silent).

If you've got any ideas, I'd love to hear them :) Thanks for nix2container!

mcdonc commented 7 months ago

Aha! Some progress. I was wrong! I didn't notice it when I tried nixpkgs-unstable and thus skopeo 1.14.1, but it behaves differently when using that version rather than the 1.13.X from nixos-23.11:

Rather than silently "succeed" without having successfully stored an image in the local docker-daemon, 1.14.1 skopeo copy errors out with a "too many levels of symbolic links" message. Here's that command (I added the --debug flag):

/nix/store/47jlk9kmbd9mbblsxqnd1jznf9nyf1qf-skopeo-1.14.1/bin/skopeo --insecure-policy copy nix:/nix/store/9cga4w2hyjf8ndj6yvs4g9x45p1kdrc5-image-climo2.json docker-daemon:climo2:latest --debug

And its output.

time="2024-01-27T20:02:26-05:00" level=debug msg="Using SQLite blob info cache at /home/chrism/.local/share/containers/cache/blob-info-cache-v1.sqlite"
time="2024-01-27T20:02:26-05:00" level=debug msg="IsRunningImageAllowed for image nix:"
time="2024-01-27T20:02:26-05:00" level=debug msg=" Using default policy section"
time="2024-01-27T20:02:26-05:00" level=debug msg=" Requirement 0: allowed"
time="2024-01-27T20:02:26-05:00" level=debug msg="Overall: allowed"
Getting image source signatures
time="2024-01-27T20:02:26-05:00" level=debug msg="Manifest has MIME type application/vnd.oci.image.manifest.v1+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json]"
Copying blob sha256:df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3
time="2024-01-27T20:02:26-05:00" level=debug msg="Checking if we can reuse blob sha256:df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3: general substitution = true, compression for MIME type \"application/vnd.oci.image.layer.v1.tar\" = true"
time="2024-01-27T20:02:26-05:00" level=debug msg="Walking filesystem: /nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1"

.. many walkings ..

time="2024-01-27T20:02:27-05:00" level=debug msg="Walking filesystem: /nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
time="2024-01-27T20:02:27-05:00" level=debug msg="No compression detected"
time="2024-01-27T20:02:27-05:00" level=debug msg="Using original blob without modification"
time="2024-01-27T20:02:27-05:00" level=debug msg="docker tarfile: input with unknown size, streaming to disk first ..."
time="2024-01-27T20:03:02-05:00" level=debug msg="... streaming done"
time="2024-01-27T20:03:02-05:00" level=debug msg="Sending as tar file df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3.tar"
Copying config sha256:34e1d32a29f43c86162826dcc13b4141e888758c16c91aef530d0e1c1789b521
time="2024-01-27T20:03:05-05:00" level=debug msg="No compression detected"
time="2024-01-27T20:03:05-05:00" level=debug msg="Compression change for blob sha256:34e1d32a29f43c86162826dcc13b4141e888758c16c91aef530d0e1c1789b521 (\"application/vnd.docker.container.image.v1+json\") not supported"
time="2024-01-27T20:03:05-05:00" level=debug msg="Using original blob without modification"
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar file 34e1d32a29f43c86162826dcc13b4141e888758c16c91aef530d0e1c1789b521.json"
Writing manifest to image destination
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar link bd00f16b609b83f7f8f1821574e58df8a3dc43286d0a65fb94fef08b48452850/layer.tar -> ../df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3.tar"
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar file bd00f16b609b83f7f8f1821574e58df8a3dc43286d0a65fb94fef08b48452850/VERSION"
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar file bd00f16b609b83f7f8f1821574e58df8a3dc43286d0a65fb94fef08b48452850/json"
time="2024-01-27T20:03:05-05:00" level=debug msg="docker-daemon: Closing tar stream"
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar file manifest.json"
time="2024-01-27T20:03:05-05:00" level=debug msg="Sending as tar file repositories"
time="2024-01-27T20:03:05-05:00" level=debug msg="docker-daemon: Waiting for status"
time="2024-01-27T20:03:13-05:00" level=debug msg="docker-daemon: sending done, status docker engine reported: mkdir /nix/store: too many levels of symbolic links"
time="2024-01-27T20:03:13-05:00" level=fatal msg="committing the finished image: docker engine reported: mkdir /nix/store: too many levels of symbolic links"

Not quite sure what to do about that. It's very odd because the same basic command works in another very similar projects (most of the same dependencies). I guess I could try a binary search, turning off features of devenv.nix by half in succession. If you've got other ideas, I'd love to hear them.

Here's the json:

{
    "version": 1,
    "image-config": {
        "Env": [
            "DEVENV_DOTFILE='/.devenv'",
            "DEVENV_PROFILE='/nix/store/3j8p1zirxnw4v60fbab7b5qknfz83xy5-devenv-profile'",
            "DEVENV_ROOT='/'",
            "DEVENV_STATE='/.devenv/state'",
            "MSSQLTOOLSSERVICE_PATH='/nix/store/5fdjh005gwmn4acq10gl8y4vjm803lla-ms-sqltoolsservice-4.11.0.10/bin'",
            "ODBCINSTINI='ywgv8ian3q8mlkydnpqqa5xndvh05pwi-odbcini'",
            "ODBCSYSINI='/nix/store'",
            "PGDATA='/.devenv/state/postgres'",
            "PGHOST='/.devenv/state/postgres'",
            "PGPORT='5432'"
        ],
        "Entrypoint": [
            "/nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
        ],
        "Cmd": [
            "/bin/bash\n"
        ]
    },
    "layers": [
        {
            "digest": "sha256:df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3",
            "size": 1639057408,
            "diff_ids": "sha256:df757211df7f24dc0f3a09850e307b57294b46de427161db2d52ad015d15ace3",
            "paths": [
                {
                    "path": "/nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1"
                },
                {
                    "path": "/nix/store/08n25j4vxyjidjf93fyc15icxwrxm2p8-libidn2-2.3.4"
                },
                {
                    "path": "/nix/store/lmidwx4id2q87f4z9aj79xwb03gsmq5j-xgcc-12.3.0-libgcc"
                },
                {
                    "path": "/nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27"
                },
                {
                    "path": "/nix/store/7vq9zymw7zk2i0k1i4nfkl6hmlx3gxji-gcc-12.3.0-libgcc"
                },
                {
                    "path": "/nix/store/myw67gkgayf3s2mniij7zwd79lxy8v0k-gcc-12.3.0-lib"
                },
                {
                    "path": "/nix/store/8xgb8phqmfn9h971q7dg369h647i1aa0-zlib-1.3"
                },
                {
                    "path": "/nix/store/q1c2flcykgr4wwg5a6h450hxbk4ch589-bash-5.2-p15"
                },
                {
                    "path": "/nix/store/11xpmmwy95396nkhih3qc3814lqhqb8f-libunistring-0.9.10"
                },
                {
                    "path": "/nix/store/1mpxs3109cjrbhmi3q1vmvc0djz102pl-libidn2-2.3.2"
                },
                {
                    "path": "/nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50"
                },
                {
                    "path": "/nix/store/k3d7ny5h682kixy91iw7sm653kxyqr8d-openssl-3.0.12"
                },
                {
                    "path": "/nix/store/1ryrrnjsxnj48hic5bk03hgvcp38sbli-zstd-1.5.5"
                },
                {
                    "path": "/nix/store/vbb1l2krfp75m5w3y2h52l0pbq6fydz8-attr-2.5.1"
                },
                {
                    "path": "/nix/store/mmy4fc1arb2ig642chgprgms27vk9hgn-acl-2.3.1"
                },
                {
                    "path": "/nix/store/wzqs107xkwdxls33skg2ma9rl7j1i68f-xz-5.4.4"
                },
                {
                    "path": "/nix/store/chkjrz6qy7i9ma62j10ij1gks571g3jn-keyutils-1.6.3-lib"
                },
                {
                    "path": "/nix/store/7d2qkskb5yg9p5lgvspapw15dkc3y3zx-libkrb5-1.20.2"
                },
                {
                    "path": "/nix/store/p40jraqsma0yb56bblyk19xmvi4d3lkl-util-linux-minimal-2.39.2-lib"
                },
                {
                    "path": "/nix/store/7b2iijic7casyjcr799iqn5i7q8ax72f-libgpg-error-1.47"
                },
                {
                    "path": "/nix/store/5i51nfixxx3p3gshkfsjj4bzp7wajwxz-ncurses-6.4"
                },
                {
                    "path": "/nix/store/injawg30ws0a6ydwbsvaaa1p0ky82pjr-gmp-with-cxx-6.3.0"
                },
                {
                    "path": "/nix/store/155qsyx1mv11fsi48nz4dlc0vh1a3drx-bzip2-1.0.8"
                },
                {
                    "path": "/nix/store/bblyj5b3ii8n6v4ra0nb37cmi3lf8rz9-coreutils-9.3"
                },
                {
                    "path": "/nix/store/ksdrjk9519bx41hbam04xpgmgg4s6q5i-nghttp2-1.57.0-lib"
                },
                {
                    "path": "/nix/store/7y0ng2kzq6zxxvjq10yi8xrda3f3yz8j-brotli-1.1.0-lib"
                },
                {
                    "path": "/nix/store/4vqq3gb4qwr1s2vwgx09bl6slqxkh6v6-libssh2-1.11.0"
                },
                {
                    "path": "/nix/store/aiv6dg0wz1k68l624x8m1vfqn9rmsdkl-curl-8.4.0"
                },
                {
                    "path": "/nix/store/s3s7gv33p88kzbgki2bprg2a1nc7jnf8-gmp-with-cxx-6.3.0"
                },
                {
                    "path": "/nix/store/4px7vzc531y10r7mmxps1s32cbshbjr2-pcre2-10.42"
                },
                {
                    "path": "/nix/store/ixvka49i2mjnxzk8l8anykc1idilws1r-libxcrypt-4.4.36"
                },
                {
                    "path": "/nix/store/aqmgny7rs6zyni27v3pxrqmpajdmdj1n-libmnl-1.0.5"
                },
                {
                    "path": "/nix/store/jbdicrrzljvl9qnzp3rdzjbr94g6mnl7-libtasn1-4.19.0"
                },
                {
                    "path": "/nix/store/8skq4a62klp1s3lldgf60ybq3h9z0m4r-libcap-2.69-lib"
                },
                {
                    "path": "/nix/store/9kd4bc8fpclpvf1vdwlbila71svyb6w1-libffi-3.4.4"
                },
                {
                    "path": "/nix/store/rx2wig5yhpbwhnqxdy4z7qivj9ln7fab-gnugrep-3.11"
                },
                {
                    "path": "/nix/store/v0q6h046vc4ji6lxk0ivb3zg9cjcgh3k-libxml2-2.11.5"
                },
                {
                    "path": "/nix/store/yrml8d394c72yffxqwpm4wl4ycfha4wr-readline-8.2p1"
                },
                {
                    "path": "/nix/store/jzjqfff4cldlm2wpld313a2s1v6r3ycb-ncurses-6.2"
                },
                {
                    "path": "/nix/store/c5giadpm71mvla6202mhz32x9criabna-zlib-1.2.11"
                },
                {
                    "path": "/nix/store/44xy8mm4jbiyp41fg2g4dwjh6aqr69wm-perl5.38.0-TimeDate-2.33"
                },
                {
                    "path": "/nix/store/xrwvr586g0i0rlw84k9dybijlkiz9ycl-perl5.38.0-URI-5.21"
                },
                {
                    "path": "/nix/store/iz7vlhvsr1lll8lzwd36ffwkzccgqkjc-nettle-3.9.1"
                },
                {
                    "path": "/nix/store/vwfzrh56pghvnmal8gmlxgyigndx6k9h-libnfnetlink-1.0.2"
                },
                {
                    "path": "/nix/store/iq5yz5c09d6d3lbb67jrndipv2x8xhgj-systemd-minimal-libs-254.6"
                },
                {
                    "path": "/nix/store/3rmfwcnhj1z3qya2lx0dlm7j129sfzfk-json-c-0.16"
                },
                {
                    "path": "/nix/store/jcq3h77q118ahjz6424k90g4093ahzjm-p11-kit-0.25.0"
                },
                {
                    "path": "/nix/store/1zf8d2qzpx32mbzfsyzd51xshaid6xqf-elfutils-0.190"
                },
                {
                    "path": "/nix/store/1whrfrfzjbbjl7hfdh34qhbzlckhpalc-libgcrypt-1.10.2"
                },
                {
                    "path": "/nix/store/r9yqs0v7af971b0rc7cmw7lsdjixwlj3-audit-3.1.2"
                },
                {
                    "path": "/nix/store/wadmyilr414n7bimxysbny876i2vlm5r-bash-5.1-p8"
                },
                {
                    "path": "/nix/store/rf6ys1x5xcyiia7wngr0k7kgsqv8rrjd-zstd-1.5.5-bin"
                },
                {
                    "path": "/nix/store/874gxh0sgwkbwb7nfbp72sn7a058b7sa-lz4-1.9.4"
                },
                {
                    "path": "/nix/store/jnkicl228rqax2r7kidg0dhwnmlfnmki-perl5.38.0-HTTP-Date-6.06"
                },
                {
                    "path": "/nix/store/cx01qk0qyylvkgisbwc7d3pk8sliccgh-glibc-2.38-27-bin"
                },
                {
                    "path": "/nix/store/xpidksbd07in3nd4sjx79ybwwy81b338-gnutar-1.35"
                },
                {
                    "path": "/nix/store/7fsdrvpwxnfb8wrb9c6wa1hzw71gn4yz-gmp-6.3.0"
                },
                {
                    "path": "/nix/store/n9s7hv5rw1fp1wfnc69iy3is12r26wfd-perl5.38.0-Clone-0.46"
                },
                {
                    "path": "/nix/store/lmcd6kbhklpmcy65r65gk1ndxg233ail-perl5.38.0-Encode-Locale-1.05"
                },
                {
                    "path": "/nix/store/fl6411hpgxys1807bj9h5w4f6idpf21r-perl5.38.0-IO-HTML-1.004"
                },
                {
                    "path": "/nix/store/58x0bb5gzgsrgmirwkd7ly1q15slnlza-perl5.38.0-LWP-MediaTypes-6.04"
                },
                {
                    "path": "/nix/store/fp3y35hsv1q080i99ll3wnyh5lmsasmi-perl5.38.0-HTTP-Message-6.45"
                },
                {
                    "path": "/nix/store/vzg3i64357bfnhd9chnij1qldbl3nqz3-libXau-1.0.11"
                },
                {
                    "path": "/nix/store/mp7kxwyb5jnndv5x53swr34c37yimljm-libXdmcp-1.1.4"
                },
                {
                    "path": "/nix/store/a4rcsjklsqqrbpxcf46p6srp8x1bjpnf-libxcb-1.16"
                },
                {
                    "path": "/nix/store/hmrg92skm94xhaippz5fqcrv992qq90b-libX11-1.8.7"
                },
                {
                    "path": "/nix/store/202iqv4bd7lh6f7fpy48p7q4d96lqdp7-gzip-1.13"
                },
                {
                    "path": "/nix/store/ik7jardq92dxw3fnz3vmlcgi9c8dwwdq-bzip2-1.0.8-bin"
                },
                {
                    "path": "/nix/store/76z4cjs7jj45ixk12yy6k5z2q2djk2jb-xz-5.4.4-bin"
                },
                {
                    "path": "/nix/store/gz6mkv6pfkkm7fb901kqvzwhh25s54c9-libossp-uuid-1.6.2"
                },
                {
                    "path": "/nix/store/sr16hq3pdq96w5kbbksnngad2czxfvys-openssl-1.1.1l"
                },
                {
                    "path": "/nix/store/sl7zsdx58kwgnf2dgj0qjm03i6d1223b-sqlite-3.36.0"
                },
                {
                    "path": "/nix/store/pvkngxfr8gqx4absf8635gvydpvsxcb4-readline-6.3p08"
                },
                {
                    "path": "/nix/store/f5d50ymyidsqkrwikkhrh4ci8sg1dvs4-expat-2.4.1"
                },
                {
                    "path": "/nix/store/csxpmhlzyhdkjir2clakkfp66yyflfvg-bzip2-1.0.6.0.2"
                },
                {
                    "path": "/nix/store/a3v9gckl2l893yn817ssfdkw1hb27ki9-xz-5.2.5"
                },
                {
                    "path": "/nix/store/6wgljzd2648yv3a908ikdprzpnaga3x7-gdbm-1.20"
                },
                {
                    "path": "/nix/store/6kgfmzx90c1a6afqnbkz6qprkzss476k-mime-types-9"
                },
                {
                    "path": "/nix/store/2z8hacx9dphisbf7syd9hvkiw4578r23-libffi-3.4.2"
                },
                {
                    "path": "/nix/store/3vx4ajfr2gy3zbl803frpabh4n85cacp-tzdata-2023c"
                },
                {
                    "path": "/nix/store/kanzv0wxiscx6ax5xx81bzlpiy2qxwxd-python3-3.6.14"
                },
                {
                    "path": "/nix/store/2gjifw9iylbkiv56jyvfi2ybyc0ivibi-icu4c-73.2"
                },
                {
                    "path": "/nix/store/h72wbvjy0ijg1df58p50biv84j18823q-libevent-2.1.12"
                },
                {
                    "path": "/nix/store/whd27ii6xl42nfzxgh6lx7iknmdfwdx7-libnl-3.7.0"
                },
                {
                    "path": "/nix/store/y37fpsmfmcqvb12bydq2h4rw8s8i4wkd-unbound-1.18.0-lib"
                },
                {
                    "path": "/nix/store/cx5hr3lcvplalqfqhawp4ishrn7ischx-dns-root-data-2019-01-11"
                },
                {
                    "path": "/nix/store/xq5alk31bmx5gy5xlb56jbgkx0ac9f81-libnetfilter_conntrack-1.0.9"
                },
                {
                    "path": "/nix/store/gifh4b7pqhkr3mn552r4jl9hj4yk7vn9-libnftnl-1.2.6"
                },
                {
                    "path": "/nix/store/bzidh53vs7h7p1r8j73blir9xpi2650h-libpcap-1.10.4"
                },
                {
                    "path": "/nix/store/h10gsyrw8z0h8cc4zg7g6v9c2mv97jxw-pcsclite-1.9.5"
                },
                {
                    "path": "/nix/store/aqqw20z31b7sxrlkbdx953sff1cfghxj-libcbor-unstable-2023-01-29"
                },
                {
                    "path": "/nix/store/4cjkx8v3z93v0l98hiwrv7960xshg9qb-libassuan-2.5.6"
                },
                {
                    "path": "/nix/store/m4ikf53wndjg3dcwsnvicagn16kal1fy-db-4.8.30"
                },
                {
                    "path": "/nix/store/14vj8gyjlp6nd6y54qqkic1ab3l9hf74-npth-1.6"
                },
                {
                    "path": "/nix/store/n1gr11z8y0kpxcrm4mk0cshz2v15g4mh-lvm2-2.03.22-lib"
                },
                {
                    "path": "/nix/store/jm1p5p48d6x6cl2c268g3dm0cxwa208p-libargon2-20190702"
                },
                {
                    "path": "/nix/store/0fyxm3fyswhp0wbw1v8hki833qgprr37-glibc-2.38-27-getent"
                },
                {
                    "path": "/nix/store/0bk337b0fpqdqjj4jsv7prmviw58fr08-gnutls-3.8.2"
                },
                {
                    "path": "/nix/store/f0hfmvdm4xy7g03q4f8zbi2q4h6d5wss-ncurses-6.4-man"
                },
                {
                    "path": "/nix/store/yjmdk8swmam79jkr4km6hjwdk4g22996-util-linux-minimal-2.39.2-login"
                },
                {
                    "path": "/nix/store/x943z8xpk8jnmfd0czqwgyp3ghgdzw20-libseccomp-2.5.4-lib"
                },
                {
                    "path": "/nix/store/x88ivkf7rmrhd5x3cvyv5vh3zqqdnhsk-bash-interactive-5.2-p15"
                },
                {
                    "path": "/nix/store/wr5z7dghb487d4w7xa5y9a2nzfziyv82-tpm2-tss-4.0.1"
                },
                {
                    "path": "/nix/store/w2lpzmvl3w613mz547zg1a99dfhxsj7z-util-linux-minimal-2.39.2-mount"
                },
                {
                    "path": "/nix/store/rnxg089s7m17w48i1rvdxdg7jv194ayy-util-linux-minimal-2.39.2-swap"
                },
                {
                    "path": "/nix/store/qnbwgvwprl13cw44p9sizyvlq7gr20z7-iptables-1.8.10"
                },
                {
                    "path": "/nix/store/mdjq8j0lx2bdsri5zycpw597z78apqfa-libmicrohttpd-0.9.71"
                },
                {
                    "path": "/nix/store/jakggpdjsardflc64id2jkvv2ba3bxbq-kexec-tools-2.0.26"
                },
                {
                    "path": "/nix/store/j0ljmgj3p2sdx4wj9fgsl4c69yjrw9fx-gnupg-2.4.1"
                },
                {
                    "path": "/nix/store/gwsk1v8pkyd8ds1gpjhv6wl95rcxfpwd-getent-glibc-2.38-27"
                },
                {
                    "path": "/nix/store/cc5vgczy2ryqg7v8lsjypwq3n534rvnd-kbd-2.6.3"
                },
                {
                    "path": "/nix/store/blimya0m8z5wfxh28nsddhlwgyrrvh4f-libfido2-1.13.0"
                },
                {
                    "path": "/nix/store/awwmn6ksi2sqvyd6gx9g2dsy2hg0cmjm-libbpf-1.2.2"
                },
                {
                    "path": "/nix/store/8hj2wknr00ihj7c0rd5wmfq8q9x9000w-kmod-31-lib"
                },
                {
                    "path": "/nix/store/1vj56kk9k01g1spfcwj6d50aal9kvpam-kmod-31"
                },
                {
                    "path": "/nix/store/18z7m0ikjz1hx3kb2z22y4017fx14v9l-linux-pam-1.5.2"
                },
                {
                    "path": "/nix/store/0p54fqkhd1zgml2rksj9dykdihfl7k88-cryptsetup-2.6.1"
                },
                {
                    "path": "/nix/store/035ar7f41f3plxl4q579ixzqgfyxrclq-libapparmor-3.1.6"
                },
                {
                    "path": "/nix/store/hd191cw5fi2k194gg5kjqh7q47k44pq7-ncurses-6.4-dev"
                },
                {
                    "path": "/nix/store/glmaxm53w5kfdbl8pl1yfyv8rmhszb7f-lz4-1.9.4-bin"
                },
                {
                    "path": "/nix/store/yna5rl3z17hxjw1n3ppljn8h3gjbdi5m-libxml2-2.11.5-bin"
                },
                {
                    "path": "/nix/store/9a4ivnm2hwggi6qjg0gcpk05f5hsw5r5-zlib-1.3-dev"
                },
                {
                    "path": "/nix/store/7ngsbi9saqi6wy61f2dnsb8qw73xagsv-systemd-254.6"
                },
                {
                    "path": "/nix/store/7j9lfm5nwwcgkdsir4m4pqs4sj0rwbq0-openssl-3.0.12-bin"
                },
                {
                    "path": "/nix/store/3kzhxyviqpwv4ghm8pj3n325vs3mdk12-perl5.38.0-HTML-Tagset-3.20"
                },
                {
                    "path": "/nix/store/7bj4kp0yf0camdasb88z4k4gy0dgm0ak-expat-2.5.0"
                },
                {
                    "path": "/nix/store/am68gj0i5hdyfhagrilimq9zylf8wdwp-linux-headers-6.5"
                },
                {
                    "path": "/nix/store/71fvzihav5227cvxisnb475vbal90457-sqlite-3.43.2"
                },
                {
                    "path": "/nix/store/mrgib0s2ayr81xv1q84xsjg8ijybalq3-glibc-2.38-27-dev"
                },
                {
                    "path": "/nix/store/bgjax7pyphpzfi7h5jdm3i4hc4q5aici-perl5.38.0-HTML-Parser-3.81"
                },
                {
                    "path": "/nix/store/w09nf1k5cwkdcs13317nl3bbmdcxgz9m-mpfr-4.2.1"
                },
                {
                    "path": "/nix/store/xpbvz28h8q08hwsfa4ghdmlgh6r87plw-postgresql-16.1-lib"
                },
                {
                    "path": "/nix/store/vdzdipmq92zcxs47qdvm2fkahglmvgjq-mailcap-2.1.53"
                },
                {
                    "path": "/nix/store/kjhznvfzdp095azz7blryagi76dhxmlr-libXext-1.3.5"
                },
                {
                    "path": "/nix/store/sbij2dnwi3vmjpzpsr1al44vmqrrzy3m-libICE-1.1.1"
                },
                {
                    "path": "/nix/store/ynik8rsd312mswcyjazjjsmq9d0xx8an-find-xml-catalogs-hook"
                },
                {
                    "path": "/nix/store/62nz2ihx8g8m3s9lipmzl8ffk13vzbax-perl5.38.0-Net-SSLeay-1.92"
                },
                {
                    "path": "/nix/store/zxk3x3ixryxixwyljz1sipxzqhg7cvvy-libkrb5-1.20.2-dev"
                },
                {
                    "path": "/nix/store/r9vjd4py76amxqdrv1p3w36nrpvld304-systemd-254.6-dev"
                },
                {
                    "path": "/nix/store/qsjbb26aqql7a0v8nhgs6fk6z3l9fn3p-readline-8.2p1-dev"
                },
                {
                    "path": "/nix/store/n1v6jm64720xs8ziya9377g97g2jd7nd-lz4-1.9.4-dev"
                },
                {
                    "path": "/nix/store/lhgj80qbyazybi5cb6kkxr8f9vj141xq-openssl-3.0.12-dev"
                },
                {
                    "path": "/nix/store/hq1zn6xmgwkawp86bhdyfsmx89ma1923-zstd-1.5.5-dev"
                },
                {
                    "path": "/nix/store/czjrzhjqdf9d6bfpw91qnji90yf5g0i9-libxml2-2.11.5-dev"
                },
                {
                    "path": "/nix/store/cs1hycx3bnfyvxkq8jv5xr0pv5d7ksj7-icu4c-73.2-dev"
                },
                {
                    "path": "/nix/store/vqs938kkhlq8x4yjyi0rg4wc31gqm1y3-unixODBC-2.3.12"
                },
                {
                    "path": "/nix/store/ffsh04qav36c283cnxsknnb99k4c76jr-postgresql-16.1"
                },
                {
                    "path": "/nix/store/w8f86kvpfp9fvazi5i4iimkrrsk5mcah-gdbm-1.23"
                },
                {
                    "path": "/nix/store/limd2s0l2fwqx555gnw235xljh83900i-openssl-3.0.12"
                },
                {
                    "path": "/nix/store/4dr83a4wn25zkb5pvsq2y3q3v586jzs9-expand-response-params"
                },
                {
                    "path": "/nix/store/w8mwxc0q6qj1d8wdp8dkpbrmavwsfl6x-perl5.38.0-Try-Tiny-0.31"
                },
                {
                    "path": "/nix/store/im29gnszjnfxyfw5pw8hva92d4y42mwb-python3.6-future-0.18.2"
                },
                {
                    "path": "/nix/store/g04k0q0l1581vk60rqnssxdh1v232zj1-python3.6-wheel-0.36.2"
                },
                {
                    "path": "/nix/store/6d7g7vlzvdxr66ppi0mcb5yp4l7hx8b0-perl5.38.0-Mozilla-CA-20230821"
                },
                {
                    "path": "/nix/store/v1jvkzadd79k62agwgq16javfl9ammcl-perl5.38.0-IO-Socket-SSL-2.083"
                },
                {
                    "path": "/nix/store/wx6hgc4gz4h18gn7jbg16kbca6zqsyq6-perl5.38.0-CGI-4.59"
                },
                {
                    "path": "/nix/store/m5hvx64acckwyf99q9cykcpq95m21igk-perl5.38.0-FCGI-0.82"
                },
                {
                    "path": "/nix/store/afz9z2iczjx13i66ymwwinwp2zvgc1ck-perl-5.38.0"
                },
                {
                    "path": "/nix/store/01r2sdjs87b81pzgih4arb9qij1yx41n-perl5.38.0-Digest-HMAC-1.04"
                },
                {
                    "path": "/nix/store/v1nmbk9bzlxdgfa4cjlsxrpxqabn0bjb-isl-0.20"
                },
                {
                    "path": "/nix/store/3ysf58a4zshpi654ya4hsfbhvmnnjrqf-libmpc-1.3.1"
                },
                {
                    "path": "/nix/store/mnlqpncq82lw07iy6yzbb7gxz78l10cy-binutils-2.40-lib"
                },
                {
                    "path": "/nix/store/wmknncrif06fqxa16hpdldhixk95nds0-strip.sh"
                },
                {
                    "path": "/nix/store/9c5qm297qnvwcf7j0gm01qrslbiqz8rs-gnused-4.9"
                },
                {
                    "path": "/nix/store/hf2gy3km07d5m0p1lwmja0rg9wlnmyr7-gcc-12.3.0"
                },
                {
                    "path": "/nix/store/f4ldf31rn0zyh4p47drs4462a1nmx9qx-sqlite-3.43.2-bin"
                },
                {
                    "path": "/nix/store/1fn92b0783crypjcxvdv6ycmvi27by0j-binutils-2.40"
                },
                {
                    "path": "/nix/store/kkcxvcj27x2lr1wkb6fnxlj3ssdx7vsm-pcre-8.45"
                },
                {
                    "path": "/nix/store/xlyhjjnmwr11bzmbky4l779gy3nip7ga-libglvnd-1.7.0"
                },
                {
                    "path": "/nix/store/6nds0a8xh53qz9adx271i17claw0qfrj-libselinux-3.3"
                },
                {
                    "path": "/nix/store/ljwksjfdcrk0ac199pnhivxai8ksj89q-libGL-1.7.0"
                },
                {
                    "path": "/nix/store/gd5sm7yrr7mfnz9gqyf9dxg1pmfs00vm-libXrender-0.9.11"
                },
                {
                    "path": "/nix/store/g878xdyhrzlhbf80jnknnp14w82j919k-glib-2.78.1"
                },
                {
                    "path": "/nix/store/85ndgar09amz954z8c4kvzfl1zk333k9-libSM-1.2.4"
                },
                {
                    "path": "/nix/store/yhg3i1kd3lj1z3jmqgzsaapzr8kf51wp-manylinux2014"
                },
                {
                    "path": "/nix/store/mci24w2n10pw4zsq9vj37dbggngr39q9-python3-3.11.7"
                },
                {
                    "path": "/nix/store/9jvj17ydc2rl6jhxnqbdpl03k6pjrd9v-libkrb5-1.20.2"
                },
                {
                    "path": "/nix/store/5pf6rhrrra110xr2xbnc3xz6fda6kjf3-openssl-1.1.1w"
                },
                {
                    "path": "/nix/store/z9xlgxp9y0rp1fihhsv75grhv9lci3x3-git-2.42.0-doc"
                },
                {
                    "path": "/nix/store/c54kyc3vn0jbq5hznimvx6wz15jxw8jq-gnu-config-2023-09-19"
                },
                {
                    "path": "/nix/store/hvi3hx26haw91f1nd5l4hp92gkpv7rg1-msodbcsql17-17.7.1.1-1"
                },
                {
                    "path": "/nix/store/dmfigbzp4b69m094wb76ky9cb0c94h6x-python3-3.11.7-env"
                },
                {
                    "path": "/nix/store/1alqjnr40dsk7cl15l5sn5y2zdxidc1v-binutils-wrapper-2.40"
                },
                {
                    "path": "/nix/store/mr1m7vvaqq522cgwm2fxglgkm1rn4irg-perl5.38.0-HTTP-Cookies-6.10"
                },
                {
                    "path": "/nix/store/hw5cmhsd546gw4gjmra1cyaka08flx01-perl5.38.0-Test-RequiresInternet-0.05"
                },
                {
                    "path": "/nix/store/b6gc735pb8fal9m883inhc7b6hqzkgp3-perl5.38.0-HTTP-CookieJar-0.014"
                },
                {
                    "path": "/nix/store/9y36vjs6xsakhyn3p742f29g25a54h6q-perl5.38.0-File-Listing-6.16"
                },
                {
                    "path": "/nix/store/9vvzj97qkxnlba148l2k0lhlgpb90rcj-perl5.38.0-Net-HTTP-6.23"
                },
                {
                    "path": "/nix/store/lyw6rjqxj51ixvw214xgz2lzixm0bzyc-postgresql.conf"
                },
                {
                    "path": "/nix/store/7274bwbvl9a7kcghb1kkbqdjqggm0wry-perl5.38.0-Test-Needs-0.002010"
                },
                {
                    "path": "/nix/store/6najzl9j69ps3d9s3qgjyl3wqf5nwkbg-perl5.38.0-Test-Fatal-0.017"
                },
                {
                    "path": "/nix/store/64399m6bbll8ylia5ysf451c3cj3n1kp-perl5.38.0-WWW-RobotRules-6.02"
                },
                {
                    "path": "/nix/store/5rdhns8kf7r5085mnccbr53y47qqm01f-perl5.38.0-HTTP-Daemon-6.16"
                },
                {
                    "path": "/nix/store/1sbvn9353xs871lavizyn2brv4rwhj84-perl5.38.0-HTTP-Negotiate-6.01"
                },
                {
                    "path": "/nix/store/2y3vj12vrpkz1xr8asll41rs2mjnz8xr-python3.6-mssql-scripter-1.0.0a23"
                },
                {
                    "path": "/nix/store/wxjk2v5vjxmpm7lkiy0mk9xcqcrhp007-gettext-0.21.1"
                },
                {
                    "path": "/nix/store/qwb7dwq5g8dzb5v166mabqcvyjf8yjzq-perl5.38.0-Net-SMTP-SSL-1.04"
                },
                {
                    "path": "/nix/store/ljn6af1x0wsfzirvq99pxrkmy8qnc39b-perl5.38.0-Authen-SASL-2.1700"
                },
                {
                    "path": "/nix/store/jjbbjykz8q3h30qqknzsynp3vj8fvq73-perl5.38.0-TermReadKey-2.38"
                },
                {
                    "path": "/nix/store/ialds7rp49iylmj11vj62anhjhfihzkm-perl5.38.0-libwww-perl-6.72"
                },
                {
                    "path": "/nix/store/hfn73jilzfivws4mxsmmljm5jn3apqli-perl5.38.0-CGI-Fast-2.16"
                },
                {
                    "path": "/nix/store/fmwq823zqsrwkv3kqlvh8p9yyglhg8rg-perl5.38.0-libnet-3.15"
                },
                {
                    "path": "/nix/store/sk1nfklnmkiw61igpg6cag7idzbfl8b0-iana-etc-20230316"
                },
                {
                    "path": "/nix/store/9z3v4n4lxhqwpi032g1j1a5adp4xxq49-pkg-config-0.29.2"
                },
                {
                    "path": "/nix/store/7np6i72hk67bn8c0vd60pkl1vzx71dm3-setup-postgres"
                },
                {
                    "path": "/nix/store/2ii0b9dq7gn6m54aix8kyr1kv8bqy9p9-ed-1.19"
                },
                {
                    "path": "/nix/store/27vf1jzj26mxh9yfc4nm4vdj3m9q9gz0-python3-3.11.6"
                },
                {
                    "path": "/nix/store/0yxmp3kyjfd3x4k9y13dxwbf5kdsvy39-perl5.38.0-FCGI-ProcManager-0.28"
                },
                {
                    "path": "/nix/store/0y5vr6r9kmfp0km62l2mlb4w6ffrqdas-perl5.38.0-HTML-TagCloud-0.38"
                },
                {
                    "path": "/nix/store/0cldli42g1whg3mgsj46ibalxc72hwc3-python3-3.6.14-env"
                },
                {
                    "path": "/nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh"
                },
                {
                    "path": "/nix/store/wzdsbnv2ba3nj91aql8jjdddfmkkdh7h-patch-shebangs.sh"
                },
                {
                    "path": "/nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh"
                },
                {
                    "path": "/nix/store/v4iswb5kwj33l46dyh2zqh0nkxxlr3mz-gnumake-4.4.1"
                },
                {
                    "path": "/nix/store/qmfxld7qhk8qxlkx1cm4bkplg1gh6jgj-file-5.45"
                },
                {
                    "path": "/nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh"
                },
                {
                    "path": "/nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh"
                },
                {
                    "path": "/nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh"
                },
                {
                    "path": "/nix/store/l974pi8a5yqjrjlzmg6apk0jwjv81yqw-findutils-4.9.0"
                },
                {
                    "path": "/nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh"
                },
                {
                    "path": "/nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh"
                },
                {
                    "path": "/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh"
                },
                {
                    "path": "/nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh"
                },
                {
                    "path": "/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh"
                },
                {
                    "path": "/nix/store/cbj1ph7zi009m53hxs90idl1f5i9i941-patch-2.7.6"
                },
                {
                    "path": "/nix/store/9vk5fqaik9zx14zpfxipg7lxkdk8c5g9-update-autotools-gnu-config-scripts-hook"
                },
                {
                    "path": "/nix/store/90h6k8ylkgn81k10190v5c9ldyjpzgl9-gcc-wrapper-12.3.0"
                },
                {
                    "path": "/nix/store/8q25nyfirzsng6p57yp8hsaldqqbc7dg-diffutils-3.10"
                },
                {
                    "path": "/nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0"
                },
                {
                    "path": "/nix/store/7wfya2k95zib8jl0jk5hnbn856sqcgfk-gawk-5.2.2"
                },
                {
                    "path": "/nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh"
                },
                {
                    "path": "/nix/store/yd58q40yb1mnn7f6p1c7sh6bwrkvsvs7-mssql-scripter"
                },
                {
                    "path": "/nix/store/pyd8yylv6y0psnivp0rqjl8m5jlmm3n9-git-2.42.0"
                },
                {
                    "path": "/nix/store/ljrkf0kwgckcy02kdlz8sjdnxb2pwjb4-pkg-config-wrapper-0.29.2"
                },
                {
                    "path": "/nix/store/clfgxa0kck0d77wfvy64rhv4b5192089-process-compose-0.69.0"
                },
                {
                    "path": "/nix/store/acsgd5wypjys1p7ssf3m8zg6ikwplmd5-start-postgres"
                },
                {
                    "path": "/nix/store/3y5bc3l0s2sn20sp2ayvkww5wckw73ns-sqlite-3.43.2-dev"
                },
                {
                    "path": "/nix/store/vvd3x6rbwjxw7rbnv7yrkd3jrk8270l6-pkg-config-0.29.2-man"
                },
                {
                    "path": "/nix/store/wgqi79zvbj6wx4rdhjk0h8vlzxzrdl5j-numactl-2.0.16"
                },
                {
                    "path": "/nix/store/pspa8i64ddmjf4h4vfkmagjlybyfblcf-pkg-config-0.29.2-doc"
                },
                {
                    "path": "/nix/store/n2mbhy95l1bqc4y6n1dhbii5l1s0i0vm-lttng-ust-2.13.6"
                },
                {
                    "path": "/nix/store/ymv18735ywf4jz24qmw0bsbj2i7g8fh5-pkg-config-wrapper-0.29.2-man"
                },
                {
                    "path": "/nix/store/yj433506cgwsnp03dxmw5ms5n2zqaf2f-postgresql-16.1-man"
                },
                {
                    "path": "/nix/store/wz3rmn84agcr0c8h5fzmi24kdhr8kz0b-gcc-12.3.0-man"
                },
                {
                    "path": "/nix/store/qcvsvw7sjgrhhb24szl2bsywwp8h6dbl-postgresql-16.1-doc"
                },
                {
                    "path": "/nix/store/ngs6rg35ml3n85l3c0p46ha5l0d83acg-gcc-12.3.0-checksum"
                },
                {
                    "path": "/nix/store/m5ny1dfx2wn2w3i79i1cfmhspf0qiq1y-sqlite-3.43.2-debug"
                },
                {
                    "path": "/nix/store/hccflxwxmdcxsywgjdnkc93a0yac1cv0-postgresql-16.1-debug"
                },
                {
                    "path": "/nix/store/glrzci7hcif29nkni1124wzwgk2mqxkq-git-2.42.0-debug"
                },
                {
                    "path": "/nix/store/js8ik0vndyb6b05hk4gl85xzdm4rc3s6-requirements.txt"
                },
                {
                    "path": "/nix/store/2cqz81547zn9x76mh1vhs6p92arl74fs-pkg-config-wrapper-0.29.2-doc"
                },
                {
                    "path": "/nix/store/04qa7dh7y6jn14pdhmz45vbn71vmj2s3-gcc-12.3.0-info"
                },
                {
                    "path": "/nix/store/ywgv8ian3q8mlkydnpqqa5xndvh05pwi-odbcini"
                },
                {
                    "path": "/nix/store/wr08yanv2bjrphhi5aai12hf2qz5kvic-stdenv-linux"
                },
                {
                    "path": "/nix/store/k23afqb8da5s4283pai9zzvrnh4bvg9n-glibc-locales-2.38-27"
                },
                {
                    "path": "/nix/store/a2008784g79p5rzk2hfpb68aib9fz7a1-devcontainer.json"
                },
                {
                    "path": "/nix/store/5fdjh005gwmn4acq10gl8y4vjm803lla-ms-sqltoolsservice-4.11.0.10"
                },
                {
                    "path": "/nix/store/3j8p1zirxnw4v60fbab7b5qknfz83xy5-devenv-profile"
                },
                {
                    "path": "/nix/store/308y1ac0ah9qclgbm32wgkbyg521wjfc-init-venv.sh"
                },
                {
                    "path": "/nix/store/l1sqg8bdg336r5hzyrqdzk7j91l9ckvl-devenv-shell-env"
                },
                {
                    "path": "/nix/store/pknxfcyfspxprc1v7dn63ig8rdqyjjx8-source"
                },
                {
                    "path": "/nix/store/jmy11m3c935yyvs4njz3s52p9azgvg6f-coreutils-full-9.3"
                },
                {
                    "path": "/nix/store/jsphhmmrrvddmmpsayr0p6rkxifsvxvc-envsubst-1.4.2"
                },
                {
                    "path": "/nix/store/6aw9inrgxrixnkbi31k9n9xf84zaxsxx-devenv-shell-env-bashrc"
                },
                {
                    "path": "/nix/store/3w84ihqymrl2pizzkccp514s7286cfqx-bash-5.2-p15-man"
                },
                {
                    "path": "/nix/store/rsn4hrk294q43786b54w2yqzfd6ypci8-root",
                    "options": {
                        "rewrite": {
                            "regex": "^/nix/store/rsn4hrk294q43786b54w2yqzfd6ypci8-root",
                            "repl": ""
                        }
                    }
                },
                {
                    "path": "/nix/store/ddxh87hs2c1k2xxh0ax7dgbjjaxfx9pf-create-paths",
                    "options": {
                        "rewrite": {
                            "regex": "^/nix/store/ddxh87hs2c1k2xxh0ax7dgbjjaxfx9pf-create-paths",
                            "repl": ""
                        }
                    }
                },
                {
                    "path": "/nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
                }
            ],
            "mediatype": "application/vnd.oci.image.layer.v1.tar"
        }
    ],
    "arch": "amd64"
}
mcdonc commented 7 months ago

I was able to track this down and make it reproducible:

https://github.com/cachix/devenv/issues/957#issuecomment-1913442505

mcdonc commented 7 months ago

As I think about this, I suppose it makes sense. It's trying to overlay "/nix" on the container image. 😭

I'm not sure there's a way to make the error more consumable. Maybe just some googleable docs somewhere say "if a container copy error message indicates 'too many levels of symbolic links', ensure that your container source dir does not contain a directory that conflicts with system paths (such as /nix)."

mcdonc commented 7 months ago

Closing this issue as it's now obvious what was happening and devenv is moving towards nonroot containers where the devenv root will be a subdirectory, so it will be much less likely to happen.