nix-du thinks live paths are dead

[cdfa]

Hi, I'm not very experienced with Nix, so the issue might be with me, but I have a problem:

nix-du say there is almost 8GiB of dead paths

$ nix-du | tred | dot -Tsvg > nix.svg
Reading dependency graph from store... 6463 nodes, 76192 edges read.
Looking for optimized paths... (this could take a long time, pass option -O0 to skip)
Size statistics for the whole store:
Alive: 2.24 GiB (0 B saved by optimisation)
Total: 10.29 GiB (not taking optimisation into account)
Computing quotient graph... 18 nodes, 35 edges.

However, nix-collect-garbage -d or sudo nix-collect-garbage -d don't clean anything.

If I look at a large path that does not fit in any of the nodes in nix.svg and try to delete it says it's still alive and can show me by which root:

$ nix-store --delete /nix/store/1lhnbf58h4vm0zw1xr0091qilw9rrc1v-ghc-8.8.4/
finding garbage collector roots...
0 store paths deleted, 0.00 MiB freed
error: cannot delete path '/nix/store/1lhnbf58h4vm0zw1xr0091qilw9rrc1v-ghc-8.8.4' since it is still alive
$ nix-store -q --roots /nix/store/1lhnbf58h4vm0zw1xr0091qilw9rrc1v-ghc-8.8.4/
/home/cdfa/Projects/ll-node-package-example/.direnv/drv -> /nix/store/i5v9rz4abd3akwl1rjm1a3xrjqbhyab5-nix-shell.drv

The size that is computed for this root is ~9MiB. I don't think that is correct.

[symphorien]

Can you compare what nix-du says with nix-store --gc --print-live | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } ' and nix-store --gc --print-dead | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } '

Can you also post the resulting graph?

[cdfa]

Numbers have changed slightly, because I suddenly had some dead paths today somehow.

$ nix-du | tred > nix.dot
Reading dependency graph from store... 5833 nodes, 68435 edges read.
Looking for optimized paths... (this could take a long time, pass option -O0 to skip)
Size statistics for the whole store:
Alive: 598.01 MiB (0 B saved by optimisation)
Total: 8.65 GiB (not taking optimisation into account)
Computing quotient graph... 13 nodes, 23 edges.
$ nix-store --gc --print-live | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } '
finding garbage collector roots...
determining live/dead paths...
9290480056
$ nix-store --gc --print-dead | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } '
finding garbage collector roots...
determining live/dead paths...
error: opening file '/home/cdfa/default.nix': No such file or directory
0

nix.dot:

digraph nixstore {
    graph [rankdir=LR];
    node [shape=box,
        style=filled
    ];
    {
        graph [rank=same];
        N1  [color="#4169E1",
            fontcolor="#ffffff",
            label="{transient} (220 B)",
            shape=tripleoctagon];
        N2  [color="#FFA800",
            fontcolor="#000000",
            label="/nix/var/nix/profiles/per-user/root/channels-2-link (228.04 MiB)",
            shape=tripleoctagon];
        N3  [color="#416AE1",
            fontcolor="#ffffff",
            label="/nix/var/nix/profiles/default-3-link (219.39 KiB)",
            shape=tripleoctagon];
        N4  [color="#4169E1",
            fontcolor="#ffffff",
            label="/nix/var/nix/profiles/per-user/cdfa/profile-11-link (8.52 KiB)",
            shape=tripleoctagon];
        N5  [color="#4169E1",
            fontcolor="#ffffff",
            label="/nix/var/nix/profiles/per-user/cdfa/profile-12-link (10.35 KiB)",
            shape=tripleoctagon];
        N6  [color="#4169E1",
            fontcolor="#ffffff",
            label="/nix/var/nix/profiles/per-user/cdfa/profile-13-link (11.90 KiB)",
            shape=tripleoctagon];
        N7  [color="#4097E4",
            fontcolor="#ffffff",
            label="/home/cdfa/Projects/ll-node-package-example/.direnv/drv (9.45 MiB)",
            shape=tripleoctagon];
    }
    N10 [color="#4170E1",
        fontcolor="#ffffff",
        label="nix-du-0.3.3 (1.19 MiB)"];
    N1 -> N10;
    N8  [color="#3DE4EC",
        fontcolor="#000000",
        label="xz-5.2.5 (32.96 MiB)"];
    N3 -> N8;
    N11 [color="#FF0000",
        fontcolor="#000000",
        label="niv-0.2.18-bin (281.96 MiB)"];
    N4 -> N11;
    N12 [color="#408BE3",
        fontcolor="#ffffff",
        label="nix-tree-0.1.1.0 (6.62 MiB)"];
    N5 -> N12;
    N6 -> N10;
    N6 -> N12;
    N9  [color="#3CEEEA",
        fontcolor="#000000",
        label="zlib-1.2.11 (37.54 MiB)"];
    N8 -> N9;
    N10 -> N8;
    N11 -> N9;
    N12 -> N11;
}

[symphorien]

Of course, I can't reproduce, so I'll need some help.

First, can you check the version of nix of:

• which nix
• systemctl show -p ExecStart nix-daemon
• ldd $(which nix-du) | grep libnix

Second, I pushed on master an option to dump the graph that nix-du reads from the nix database before any processing.

The best course of action would be to

• run nix-du and dump the store graph: nix-du --dump=/tmp/foo.dot >/dev/null
• locate an alive store path heavier than what nix-du reports as alive (for example ghc):

  nix-store --gc --print-live | xargs nix path-info -s | sort -r -n -k2 | head

• check that the store path appears correctly in foo.dot That part is more tricky, because the store will be a large graph and most probably cannot be rendered by graphviz. If it is, pass it through tred, and open it with zgrviewer. It has a tool to show the transitive parents of you store path, and then you can check if some of them are gc roots (octagons). Also check that gc roots are correct: you can get the correct gc roots with sudo nix-store --gc --print-roots

[cdfa]

Of course :)

nix --version: nix (Nix) 2.3.8

$ systemctl show --no-pager -p ExecStart nix-daemon
ExecStart={ path=/nix/store/l0jnrw6cvf1rzh99bgrhnja2d3i0g82r-nix-2.3.8/bin/nix-daemon ; argv[]=nix-daemon --daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
$ ldd $(which nix-du) | grep libnix
libnixstore.so => /nix/store/l0jnrw6cvf1rzh99bgrhnja2d3i0g82r-nix-2.3.8/lib/libnixstore.so (0x00007fa18de9c000)
libnixmain.so => /nix/store/l0jnrw6cvf1rzh99bgrhnja2d3i0g82r-nix-2.3.8/lib/libnixmain.so (0x00007fa18de68000)
libnixutil.so => /nix/store/l0jnrw6cvf1rzh99bgrhnja2d3i0g82r-nix-2.3.8/lib/libnixutil.so (0x00007fa18ddb1000)
$ nix-store --gc --print-live | xargs nix path-info -s | sort -r -n -k2 | head
finding garbage collector roots...
determining live/dead paths...
/nix/store/0zfnq2gnjxlrwc2fr1frv1m1x70plgz5-ghc-8.8.4                                                         1670437192
/nix/store/1lhnbf58h4vm0zw1xr0091qilw9rrc1v-ghc-8.8.4                                                         1670436128
/nix/store/jrkqkdvvjvamhhy182q4gk4ckcmkg4ji-rustc-1.47.0                                                      712172968
/nix/store/r9anmfg7pj634rq2q0fw09srjg71im24-ghc-8.8.4-doc                                                     284367432
/nix/store/hss8cz1a6w18bwcjmfhajl2cjlkf9jsy-ghc-8.8.4-doc                                                     284367432
/nix/store/n8b40m7r4mhivv1nqvljngbw2cx21mnb-node-sources                                                      282855760
/nix/store/g8wz701j2z9yf5vj0nxx306zlhskynnq-node-sources                                                      276672656
/nix/store/psshq3ajzz2j0z5260hl47i76i9p7akg-llvm-10.0.1                                                       265842216
/nix/store/746dfcg71yl9kql0chqg7za5fkhf2wci-glibc-locales-2.31                                                219734944
/nix/store/r68k5h3k9zynrqyi9si09cv92bjfjm6m-haskell-src-exts-1.23.1                                           147854576

It took my laptop a while, but it did manage to render an svg graph after a while. It looks really cool! Furthermore, it seems to be correct. ghc-8.4.4 does appear and is linked to a valid root. What I can't figure out, is why nix-du then thinks it's dead. I uploaded the file here.

Please let me know if I can help further :)

[symphorien]

I tried to have a look at the svg but it's not very handy. The dot format would be more useful because there is a tool to highlight the ancestors of a node in zgrviewer (only zgrviewer is able to zoom enough it seems while still being able to search for text.

[cdfa]

Ah okay, here is the .dot file.

[symphorien]

the link needs a login to access it

[symphorien]

(if you zip it, you can put it as an attachement to github, without using a third party sharing medium)

[cdfa]

Oh derp. Didn't know about the zip file, thanks for the tip! nix-dump.zip

[symphorien]

I had a look: you have two copies of ghc-8.8.4. One is pulled by hnix-0.9.1, the other by niv-0.2.18. Both have no parents which suggest they are really dead. Can you check that nix agrees: ls /nix/store/*hnix-0.9.1 to find the store path and then nix-store --query --roots path and nix-store --query --referers path ?

[cdfa]

Hmm, not entirely sure what to make of this. It belongs to the correct root but has no referers?

$ nix-store --query --roots /nix/store/0l5nkm5v7l3s6fwvhc8p07c3ckl78nyz-hnix-0.9.1/
/home/cdfa/Projects/ll-node-package-example/.direnv/drv -> /nix/store/i5v9rz4abd3akwl1rjm1a3xrjqbhyab5-nix-shell.drv
$ nix-store --query --referers /nix/store/0l5nkm5v7l3s6fwvhc8p07c3ckl78nyz-hnix-0.9.1/
/nix/store/0l5nkm5v7l3s6fwvhc8p07c3ckl78nyz-hnix-0.9.1

[symphorien]

It belongs to a root which points to a drv, and the drv cannot depend on realised outputs, so it does not make sense. Would you mind opening an issue on nix?

[cdfa]

Okay, so the path is really dead and nix-du is correct? I'm still not understanding completely, but I created an issue. Please feel free to suggest a better name and add to it :)

[cdfa]

From the issue on nix, the inconsistency seems to be caused by the keep-outputs option. Is that something nix-du takes into account?

[symphorien]

No, nix-du does not take into account either keep-outputs or keep-derivations. I pushed a fix to master, can you test it?

Note that I wrote nix-du with the assumption that the store could not contain cycles (except self loops). But with both keep-outputs and keep-derivations set to true, a derivation and its output constistute a cycle, and I don't know what will happen...

[cdfa]

It's a lot better, but still not quite correct I think:

Log

Note: it's different from last time because I just had to make space for a new project ``` ❰cdfa❙~❱✔≻ nix-du | tred | dot -Tsvg > nix.svg Reading dependency graph from store... 10865 nodes, 163878 edges read. Looking for optimized paths... (this could take a long time, pass option -O0 to skip) Size statistics for the whole store: Alive: 15.00 GiB (39.90 KiB saved by optimisation) Total: 16.80 GiB (not taking optimisation into account) Computing quotient graph... 39 nodes, 149 edges. ❰cdfa❙~❱✔≻ nix-store --gc --print-dead | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } ' finding garbage collector roots... determining live/dead paths... 372163528 ❰cdfa❙~❱✔≻ nix-store --gc --print-live | xargs nix path-info -s | awk ' BEGIN { a = 0; } { a+=$2; } END { print a; } ' finding garbage collector roots... determining live/dead paths... 17667416504 ❰cdfa❙~❱✔≻ nix-collect-garbage -d removing old generations of profile /nix/var/nix/profiles/per-user/cdfa/profile removing generation 20 finding garbage collector roots... deleting garbage... deleting '/nix/store/zx5ihshd96i5k68zf76v8rrpjqz277qp-frugel-0.0.1.drv' deleting '/nix/store/12abm6rad0n5dkbvkpbgsrf3a5h4wpcm-cabal2nix-frugel.drv' deleting '/nix/store/s01nc6s25j54mi9jjv7l4d07p18xykxi-cabal2nix-frugel' deleting '/nix/store/wf8rmxdjqashb6zz1344f9kx4q2w9gwm-frugel' deleting '/nix/store/mb8a9y9jrbd5fhww1s6rhz1y2q41b05s-pre-commit-hooks.nix-src' deleting '/nix/store/fgj4bcijy63j4qj5dxk0hkvlqjvzxmsk-cabal2nix-frugel' deleting '/nix/store/j23crlwaphsladnis7gqp2nb3l9rz6l1-nixpkgs-src' deleting '/nix/store/7b3xjk6a8sxybbadnnnid1gi57lp2xc3-frugel-0.0.1.drv' deleting '/nix/store/lvm00jgyz6qkyj76r7xdf28am1wc3jql-cabal2nix-frugel' deleting '/nix/store/p1gxyqlbn29d31hxx42x0dj2m5hi3hjk-cabal2nix-frugel.drv' deleting '/nix/store/dzh3psblphqdy1a5n4j1y69jssndsi2p-frugel' deleting '/nix/store/551z2nlqayn79vy1l44vgi0a4mjka50h-user-environment' deleting '/nix/store/wqw9qzidzi1jy9rbv2zaf6gimzg0r3gq-user-environment.drv' deleting '/nix/store/hxysnqz2iw0slp34rjfyxhqvhj67wkjj-env-manifest.nix' deleting '/nix/store/wypm9clfiwrf5wkd59ik03d9yfj25n60-rust_nix-du-0.3.1' deleting '/nix/store/ig6zskd648vc15jx4yvxmga4zzmr4p32-miso-src' deleting '/nix/store/rvf06490xw5cjmmsn12bc6hrc121594k-pre-commit-hooks.nix-src' deleting '/nix/store/v7y2yj9jj0whhr0j8q146zp9z4mcdpqh-ghc-shell-for-frugel-0.0.1.drv' deleting '/nix/store/g2fk013jcnjs8ddhs2jcjbqmbchfn38z-master.tar.gz' deleting '/nix/store/8vi00pwcg44r14a62gg7g0vm6mlyvs8c-source' deleting '/nix/store/smvan4qznfgg79w1s6wjw1zdbszkynav-source' deleting '/nix/store/m2kkwl9fyqbkbq3jj19qx554w9y5rdw2-frugel-0.0.1.drv' deleting '/nix/store/hlxg536dw3d1nklw6krrvmsalljv6qbg-cabal2nix-frugel.drv' deleting '/nix/store/lnvss41ni9xx52zflyjjq6f8cbnh0b7y-frugel' deleting '/nix/store/5603xjfhrmrqfl57dmcw24w8mkvnqlhk-nixpkgs-unstable-src' deleting '/nix/store/pak6n8g7z50zpbl3922bg1ppyr6hldl6-miso-src' deleting '/nix/store/2an4vvqgsyjg4c123kwznng7vsa1smrb-cabal2nix-frugel' deleting '/nix/store/77r6345vmjqpr47qn4ga59x6lfhpnkw0-frugel-0.0.1.drv' deleting '/nix/store/fzvsjqskxbdl4k70hpc2g96pl5yzsrw6-cabal2nix-frugel.drv' deleting '/nix/store/74qln598gxdnhgpg86d1lnrq5c0pxzrw-frugel' deleting '/nix/store/xming3ng5wh2qbh0zifi8js8066g8a6q-master.tar.gz' deleting '/nix/store/5m40964dxfp0nx61ib5i731qbjlrmkqq-ghc-shell-for-frugel-0.0.1.drv' deleting '/nix/store/dyn8bgqn310d1g3j2my8y2v1zsly94bp-git.drv' deleting '/nix/store/9r1vhvri4ba913g4vyg5jqrhg6pb9amn-source' deleting '/nix/store/00lgg99nszykdzaiakwd243i30syi14d-nixpkgs-src' deleting '/nix/store/bwf8kqwsfsj4synbzpm94pgi04sh6n1p-git' deleting '/nix/store/afzbi6rh60qn7vlcw2ydkd5lsdyprk12-ghc-shell-for-frugel-0.0.1.drv' deleting '/nix/store/6l7ffzgbww1pjzhip53p83h3rnmikc87-source.drv' deleting '/nix/store/if4jzwdz8vhjwbsffgi3qd9njl4fyvmg-nixpkgs-unstable-src' deleting '/nix/store/trash' deleting unused links... note: currently hard linking saves -0.00 MiB 39 store paths deleted, 329.92 MiB freed ❰cdfa❙~❱✔≻ nix-du >/dev/null Reading dependency graph from store... 10825 nodes, 163220 edges read. Looking for optimized paths... (this could take a long time, pass option -O0 to skip) Size statistics for the whole store: Alive: 15.00 GiB (39.90 KiB saved by optimisation) Total: 16.45 GiB (not taking optimisation into account) Computing quotient graph... 33 nodes, 121 edges. ❰cdfa❙~❱✔≻ nix-du --dump=/tmp/nix-dump2^C ❰cdfa❙~❱✔≻ nix-du --dump=/tmp/nix-dump2.dot >/dev/null Reading dependency graph from store... 10825 nodes, 163220 edges read. Dumping dependency graph to /tmp/nix-dump2.dot... done Looking for optimized paths... (this could take a long time, pass option -O0 to skip) Size statistics for the whole store: Alive: 15.00 GiB (39.90 KiB saved by optimisation) Total: 16.45 GiB (not taking optimisation into account) Computing quotient graph... 33 nodes, 121 edges. ❰cdfa❙~❱✔≻ cat /tmp/nix-dump2.dot | tred > nix-dump2.dot warning: nixstore has cycle(s), transitive reduction not unique cycle involves edge N4244 -> N975 ```

Zip with a dump and output svg: nix-dump2.zip

[symphorien]

Thanks for testing.Which commit of nix-du did you use to obtain this ? I see that a store path ghcjs-75c61af has no referrers nor references, can you check what nix-store -query --references and --referrers and --roots says about this store path ?

[symphorien]

bump

[cdfa]

Apologies for taking so long to respond. I've been very busy, but I was also having some nix issues. With regards to your last comment: I'm not sure how to find the commit of nix-du I used. I did nix-env -if https://github.com/symphorien/nix-du/archive/master.tar.gz at some point and collected garbage after, so I think the archive is deleted. It seems I have also deleted ghcjs-75c61af now. Thinking about it, I also might have had keep-outputs = false with the last test, so this might change your interpretation of the results.

The nix issues I was mentioning have to do with nix path-info telling me some paths are invalid, which I though might weird results, so I would garbage collect and wait a while before I would have some stuff to garbage collect again. I just checked again and there were no invalid paths, but apparently keep-outputs = true was not in effect immediately, so the nix and nix-du output don't match 😓. I'll try to run another test soon.

[symphorien]

I tagged a new release, I think that fixes the issue, feel free to reopen if needed.