Crash with Nix 2.14.x

[milloni]

Describe the bug

The Nix 2.14.x release seems to cause flakes to evaluate eagerly, as in for example you can not run an app from a flake if a package does not evaluate despite it not being needed to run the app.

We're seeing this issue with the horizon-gen-nix app in horizon-platform - see reproduction steps below

This was originally brought up here: https://discourse.nixos.org/t/nix-2-14-0-released/25900/14

Steps To Reproduce

1. Clone horizon-platform
2. Run rm pkgs/bimap.nix (Note: removing any file from pkgs will do. Removing a file from pkgs forces horizon-gen-nix to regenerate it`
3. Run nix run .#horizon-gen-nix

Following the steps above cause nix to crash with following output:

[milloni@gazorpazorp:~/code/horizon/horizon-platform]$ nix run .#horizon-gen-nix --show-trace
warning: Git tree '/home/milloni/code/horizon/horizon-platform' is dirty
warning: ignoring untrusted substituter 'https://horizon.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
error:
       … while calling anonymous lambda

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/types.nix:546:29:

          545|       merge = loc: defs:
          546|         zipAttrsWith (name: defs:
             |                             ^
          547|           let merged = mergeDefinitions (loc ++ [name]) elemType defs;

       … while calling anonymous lambda

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:743:28:

          742|         # Process mkMerge and mkIf properties.
          743|         defs' = concatMap (m:
             |                            ^
          744|           map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))

       … while evaluating definitions from `/nix/store/8zrjcl2ssb4b83m7wpqyvmj004h7bx6p-source/modules/transposition.nix':

       … from call site

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:744:137:

          743|         defs' = concatMap (m:
          744|           map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))
             |                                                                                                                                         ^
          745|         ) defs;

       … while calling 'dischargeProperties'

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:815:25:

          814|   */
          815|   dischargeProperties = def:
             |                         ^
          816|     if def._type or "" == "merge" then

       … while calling anonymous lambda

         at /nix/store/8zrjcl2ssb4b83m7wpqyvmj004h7bx6p-source/modules/transposition.nix:59:22:

           58|           mapAttrs
           59|             (system: v: v.${attrName})
             |                      ^
           60|             config.allSystems

       … while calling 'g'

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/attrsets.nix:535:19:

          534|           g =
          535|             name: value:
             |                   ^
          536|             if isAttrs value && cond value

       … from call site

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/attrsets.nix:538:20:

          537|               then recurse (path ++ [name]) value
          538|               else f (path ++ [name]) value;
             |                    ^
          539|         in mapAttrs g;

       … while calling anonymous lambda

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:270:72:

          269|           # For definitions that have an associated option
          270|           declaredConfig = mapAttrsRecursiveCond (v: ! isOption v) (_: v: v.value) options;
             |                                                                        ^
          271|

       … while evaluating the option `perSystem.x86_64-linux.packages':

       … while calling anonymous lambda

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:743:28:

          742|         # Process mkMerge and mkIf properties.
          743|         defs' = concatMap (m:
             |                            ^
          744|           map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))

       … while evaluating definitions from `/nix/store/zv9y4d81s91sw7lkkcv7hps0lsddf0x0-source/flake.nix, via option perSystem':

       … from call site

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:744:137:

          743|         defs' = concatMap (m:
          744|           map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))
             |                                                                                                                                         ^
          745|         ) defs;

       … while calling 'dischargeProperties'

         at /nix/store/dxfmn2pdvvpiark08jlwc8s8gkhviiw1-source/lib/modules.nix:815:25:

          814|   */
          815|   dischargeProperties = def:
             |                         ^
          816|     if def._type or "" == "merge" then

       … from call site

         at /nix/store/zv9y4d81s91sw7lkkcv7hps0lsddf0x0-source/flake.nix:70:22:

           69|
           70|           packages = filterAttrs
             |                      ^
           71|             (n: v: v != null

       … while calling 'filterAttrs'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/lib/attrsets.nix:305:5:

          304|     # The attribute set to filter
          305|     set:
             |     ^
          306|     listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set));

       … while calling anonymous lambda

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/lib/attrsets.nix:306:29:

          305|     set:
          306|     listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set));
             |                             ^
          307|

       … from call site

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/lib/attrsets.nix:306:62:

          305|     set:
          306|     listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set));
             |                                                              ^
          307|

       … while calling anonymous lambda

         at /nix/store/zv9y4d81s91sw7lkkcv7hps0lsddf0x0-source/flake.nix:71:17:

           70|           packages = filterAttrs
           71|             (n: v: v != null
             |                 ^
           72|               && builtins.typeOf v == "set"

       … from call site

         at /nix/store/zv9y4d81s91sw7lkkcv7hps0lsddf0x0-source/initial-packages.nix:84:20:

           83|
           84|   abstract-deque = self.callPackage (./pkgs/abstract-deque.nix) { };
             |                    ^
           85|

       … while calling 'callPackage'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:125:22:

          124|   defaultScope = mkScope self;
          125|   callPackage = drv: args: callPackageWithScope defaultScope drv args;
             |                      ^
          126|

       … from call site

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:125:28:

          124|   defaultScope = mkScope self;
          125|   callPackage = drv: args: callPackageWithScope defaultScope drv args;
             |                            ^
          126|

       … while calling 'callPackageWithScope'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:81:37:

           80|   # here `bar` is a manual argument.
           81|   callPackageWithScope = scope: fn: manualArgs:
             |                                     ^
           82|     let

       … from call site

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:108:8:

          107|       };
          108|     in lib.makeOverridable drvScope (auto // manualArgs);
             |        ^
          109|

       … while calling 'makeOverridable'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/lib/customisation.nix:78:24:

           77|   */
           78|   makeOverridable = f: origArgs:
             |                        ^
           79|     let

       … from call site

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/lib/customisation.nix:80:16:

           79|     let
           80|       result = f origArgs;
             |                ^
           81|

       … while calling 'drvScope'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:97:18:

           96|       # this wraps the `drv` function to add `scope` and `overrideScope` to the result.
           97|       drvScope = allArgs: ensureAttrs (drv allArgs) // {
             |                  ^
           98|         inherit scope;

       … from call site

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:97:27:

           96|       # this wraps the `drv` function to add `scope` and `overrideScope` to the result.
           97|       drvScope = allArgs: ensureAttrs (drv allArgs) // {
             |                           ^
           98|         inherit scope;

       … while calling 'ensureAttrs'

         at /nix/store/ld350gbq2w5654yw4q95kxhxp4569zi6-source/pkgs/development/haskell-modules/make-package-set.nix:94:21:

           93|       # Converts a returned function to a functor attribute set if necessary
           94|       ensureAttrs = v: if builtins.isFunction v then { __functor = _: v; } else v;
             |                     ^
           95|

       error: getting status of '/nix/store/zv9y4d81s91sw7lkkcv7hps0lsddf0x0-source/pkgs/abstract-deque.nix': No such file or directory

Expected behavior

horizon-gen-nix should complete with no fatal errors. pkgs/bimap.nix should be regenerated.

nix-env --version output

nix-env (Nix) 2.14.1

Additional context

N/A

Priorities

N/A

[roberth]

This trace is hard to understand.

• It does not start with why it's evaluating something. Presumably the CLI is evaluating some flake attribute, but this information is missing from the trace.
  • It is too easy not to add this information when consuming the evaluator, but a wrapper API could add this information automatically: https://github.com/NixOS/nix/issues/5415
• The trace has too much irrelevant information
  • https://github.com/NixOS/nix/issues/7553
• /nix/store/<hash>-source does not tell me which flake it is. I believe lazy-trees may improve the presentation of flake source paths.

  • 6530