possible regression: "stack overflow (possible infinite recursion)"

[NobbZ]

Describe the bug

When trying to nix build gitlab:nobodyinperson/annextimelog with a recent build from master, an error is printed:

$ nix build gilab:nobodyinperson/annextimelog
copying '/nix/store/p418a043fs5smlx6cgj9z4q57kxlmym8-source' to the storeerror: stack overflow (possible infinite recursion)

2.18.1 worked fine and produces a proper result link. I am currently trying to bisect the issue. Which is not quite easy, as the error only occurs, when parts of the result are not available in the store, and I have not yet figured out what exactly needs to get removed, so I have to do a full GC each cycle (and building nix takes another huge amount of time)

Steps To Reproduce

1. nix build gilab:nobodyinperson/annextimelog with a nix "nightly"
2. See error

Expected behavior

A result links is created, like it got created during the 2.18.1

nix-env --version output

$ nix --version
nix (Nix) 2.20.0pre20231227_a21c762

Additional context

So far this one flake is the only that made me have the problem.

Priorities

Add :+1: to issues you find important.

[NobbZ]

bisection ended with ea95327e72f5781295417b0eae46a5e351bebebd being the first bad commit.

commit ea95327e72f5781295417b0eae46a5e351bebebd
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Nov 30 16:16:17 2023 +0100

    Move restricted/pure-eval access control out of the evaluator and into the accessor

 src/libcmd/installables.cc             |   7 +-
 src/libexpr/eval.cc                    | 103 ++++++----------------------
 src/libexpr/eval.hh                    |  25 +++----
 src/libexpr/parser.y                   |  19 +++++-
 src/libexpr/primops.cc                 | 119 +++++++++++++++------------------
 src/nix-build/nix-build.cc             |   7 +-
 src/nix-instantiate/nix-instantiate.cc |   2 +-
 tests/functional/restricted.sh         |   4 +-
 8 files changed, 115 insertions(+), 171 deletions(-)

git bisect log
git bisect start
# status: waiting for both good and bad commits
# good: [f5f4de6a550327b4b1a06123c2e450f1b92c73b6] Merge pull request #9086 from NixOS/backport-9081-to-2.18-maintenance
git bisect good f5f4de6a550327b4b1a06123c2e450f1b92c73b6
# status: waiting for bad commit, 1 good commit known
# bad: [a21c762dab365049b77af95355ee4236d173e216] Merge pull request #9666 from unblevable/dervation-typo
git bisect bad a21c762dab365049b77af95355ee4236d173e216
# good: [d40e91440d53a982b8c72b176c14f2a39c3e04d1] Merge pull request #9002 from NixOS/release-notes
git bisect good d40e91440d53a982b8c72b176c14f2a39c3e04d1
# good: [77dceb2844276217bff321d80f601297f3581530] Drop obsolete assert and cast
git bisect good 77dceb2844276217bff321d80f601297f3581530
# good: [5fe2accb754249df6cb8f840330abfcf3bd26695] fix up release note
git bisect good 5fe2accb754249df6cb8f840330abfcf3bd26695
# bad: [ff992f8b4604c28c479a1511cb789dd912494172] Merge remote-tracking branch 'upstream/master' into package-nix
git bisect bad ff992f8b4604c28c479a1511cb789dd912494172
# bad: [d4f6b1d38bfd0d5bb0ae6df9b98e184fe5da58b8] Merge pull request #9497 from edolstra/move-access-control
git bisect bad d4f6b1d38bfd0d5bb0ae6df9b98e184fe5da58b8
# skip: [5334c9c792a208db4d3824e88019a626ded1b65d] HashType: Rename to HashAlgorithm
git bisect skip 5334c9c792a208db4d3824e88019a626ded1b65d
# bad: [504e4fc4576dc6a4cd5c083a3bf7b80dfb0ca220] CanonPath: Support std::hash
git bisect bad 504e4fc4576dc6a4cd5c083a3bf7b80dfb0ca220
# bad: [43d9fb6cf180c421be17b4247f5dd032cf4843f5] Remove InputAccessor::root()
git bisect bad 43d9fb6cf180c421be17b4247f5dd032cf4843f5
# bad: [305939655a6cd680997981ca6077d4ce7f957984] Remove superfluous use of hasAccessControl()
git bisect bad 305939655a6cd680997981ca6077d4ce7f957984
# bad: [ea95327e72f5781295417b0eae46a5e351bebebd] Move restricted/pure-eval access control out of the evaluator and into the accessor
git bisect bad ea95327e72f5781295417b0eae46a5e351bebebd
# first bad commit: [ea95327e72f5781295417b0eae46a5e351bebebd] Move restricted/pure-eval access control out of the evaluator and into the accessor

[tomberek]

• Successful result in 2.19.2
• Failure on 2.21.0

instantiated 'tzdata-2023.4-py2.py3-none-any.whl' -> '/nix/store/4zx8naw73qvj754mklhyb0rl6spq39mh-tzdata-2023.4-py2.py3-none-any.whl.drv'
performing daemon worker op: 7
instantiated 'python3.11-tzdata-2023.4' -> '/nix/store/nm62rq4kw92wriiry3ym2zbjb7cq4ixp-python3.11-tzdata-2023.4.drv'
source path '/nix/store/30744h1jcq0v44i1gwliy0rbv8ibz2im-source' is uncacheable
copying '/nix/store/30744h1jcq0v44i1gwliy0rbv8ibz2im-source' to the store...
performing daemon worker op: 7
evaluating file '/nix/store/fn4i255gjsbqhza45ih8mgxgx47zan8i-source/pkgs/build-support/nix-gitignore/default.nix'
evaluating derivation 'gitlab:nobodyinperson/annextimelog#packages.x86error: stack overflow (possible infinite recursion)

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-03-25-nix-team-meeting-minutes-133/42167/1

[sweenu]

Since I updated to the latest nixos-unstable a few days ago, I seem to have this issue with one specific program I wrote: https://github.com/sweenu/ig-story-fetcher When I try to deploy it now and run a nix flake check, I get: copying '/nix/store/zw5cyffjf8vi761gf7d3lw9czxjvp6sz-source' to the storeerror: stack overflow (possible infinite recursion)

And if I run nix flake check --no-build:

error:
       … while checking flake output 'nixosConfigurations'

       … while checking the NixOS configuration 'nixosConfigurations.najdorf'
         at /nix/store/94gag39fil45c81x34rragrb3radkhyh-source/lib/mkFlake.nix:132:7:
          131|     {
          132|       ${host.output}.${reverseDomainName} = host.builder ({
             |       ^
          133|         inherit (host) system;

       … while calling the 'head' builtin
         at /nix/store/amxd2p02wx78nyaa4bkb0hjvgwhz1dq7-source/lib/attrsets.nix:1541:11:
         1540|         || pred here (elemAt values 1) (head values) then
         1541|           head values
             |           ^
         1542|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: stack overflow; max-call-depth exceeded
       at /nix/store/kzw8aq83xcx154dbwkld866hc18am5ys-source/lib.nix:61:20:
           60|     in
           61|     lib.optionals (builtins.pathExists path && builtins.toString path != "/" && ! isGitRoot) (findGitIgnores parent) ++ gitIgnores;
             |                    ^
           62|

The error seem to be related to nix-gitignore.

[thufschmitt]

Triaged during the maintainers meeting today. Reproduced by Eelco. Eelco will look at it.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-03-nix-team-meeting-135/42962/1

[Atry]

Changing mkPoetryApplication { projectDir = self; } to mkPoetryApplication { projectDir = ./.; } would suppress the error.

[NobbZ]

There are slight different semantics with that.

And in general I'd prefer to keep the former possible to avoid double copy and /nix/store/$newhash-$oldhash-source

[edolstra]

This is fixed in #10505.

Note: the infinite recursion in findGitIgnores in poetry2nix happens because it has a stop condition builtins.toString path != "/" that never happens, because path will be an uncanonicalized path like /nix/store/<bla>/../../... However, previously that wasn't an issue because pathExists "/nix/store" returned false and caused it to bail out.

[Ericson2314]

https://github.com/NixOS/nix/pull/10505#issuecomment-2069580805 per this comment, I think builtins.pathExists "/nix/store" == true might well be better behavior for pure eval mode. There is definitely a behavior change, but I am apprehensive about calling this a bug to be fixed.