Open gonzaloetjo opened 7 months ago
Here is an easier repro:
[chrism@optinix:~/projects]$ git clone git@github.com:gonzaloetjo/foundry-devenv.git
Cloning into 'foundry-devenv'...
remote: Enumerating objects: 34, done.
remote: Counting objects: 100% (34/34), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 34 (delta 15), reused 29 (delta 13), pack-reused 0
Receiving objects: 100% (34/34), 13.43 KiB | 764.00 KiB/s, done.
Resolving deltas: 100% (15/15), done.
[chrism@optinix:~/projects]$ nix profile install --accept-flake-config github:cachix/devenv/v1.0.3
[chrism@optinix:~/projects]$ cd foundry-devenv/
[chrism@optinix:~/projects/foundry-devenv]$ git log --oneline|head -1
352f845 feat: update
[chrism@optinix:~/projects/foundry-devenv]$ devenv shell
• Building shell ...
• Using Cachix: devenv
error:
… while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:9:12:
8|
9| strict = derivationStrict drvAttrs;
| ^
10|
… while evaluating derivation 'devenv-shell'
whose name attribute is located at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/pkgs/stdenv/generic/make-derivation.nix:331:7
… while evaluating attribute 'DEVENV_PROFILE' of derivation 'devenv-shell'
… while evaluating derivation 'devenv-profile'
whose name attribute is located at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/pkgs/stdenv/generic/make-derivation.nix:331:7
… while evaluating attribute 'passAsFile' of derivation 'devenv-profile'
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/pkgs/build-support/trivial-builders/default.nix:69:9:
68| inherit buildCommand name;
69| passAsFile = [ "buildCommand" ]
| ^
70| ++ (derivationArgs.passAsFile or [ ]);
… from call site
at «github:cachix/devenv/a7939d53e20cd2f44b209fac0aaa5389a9078127»/src/modules/top-level.nix:15:13:
14| name = "devenv-profile";
15| paths = lib.flatten (builtins.map drvOrPackageToPaths config.packages);
| ^
16| ignoreCollisions = true;
… while calling 'flatten'
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/lists.nix:383:13:
382| */
383| flatten = x:
| ^
384| if isList x
… from call site
at «github:cachix/devenv/a7939d53e20cd2f44b209fac0aaa5389a9078127»/src/modules/top-level.nix:15:59:
14| name = "devenv-profile";
15| paths = lib.flatten (builtins.map drvOrPackageToPaths config.packages);
| ^
16| ignoreCollisions = true;
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/attrsets.nix:1171:18:
1170| mapAttrs
1171| (name: value:
| ^
1172| if isAttrs value && cond value
… from call site
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/attrsets.nix:1174:18:
1173| then recurse (path ++ [ name ]) value
1174| else f (path ++ [ name ]) value);
| ^
1175| in
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/modules.nix:242:72:
241| # For definitions that have an associated option
242| declaredConfig = mapAttrsRecursiveCond (v: ! isOption v) (_: v: v.value) options;
| ^
243|
… while evaluating the option `packages':
… from call site
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/modules.nix:846:59:
845| if isDefined then
846| if all (def: type.check def.value) defsFinal then type.merge loc defsFinal
| ^
847| else let allInvalid = filter (def: ! type.check def.value) defsFinal;
… while calling 'merge'
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/types.nix:552:20:
551| check = isList;
552| merge = loc: defs:
| ^
553| map (x: x.value) (filter (x: x ? value) (concatLists (imap1 (n: def:
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/types.nix:553:35:
552| merge = loc: defs:
553| map (x: x.value) (filter (x: x ? value) (concatLists (imap1 (n: def:
| ^
554| imap1 (m: def':
… from call site
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/types.nix:553:38:
552| merge = loc: defs:
553| map (x: x.value) (filter (x: x ? value) (concatLists (imap1 (n: def:
| ^
554| imap1 (m: def':
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/lists.nix:334:29:
333| */
334| imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
| ^
335|
… from call site
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/lists.nix:334:32:
333| */
334| imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
| ^
335|
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/types.nix:554:21:
553| map (x: x.value) (filter (x: x ? value) (concatLists (imap1 (n: def:
554| imap1 (m: def':
| ^
555| (mergeDefinitions
… while calling anonymous lambda
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/modules.nix:824:28:
823| # Process mkMerge and mkIf properties.
824| defs' = concatMap (m:
| ^
825| map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))
… while evaluating definitions from `/nix/store/virtual0000000000000000000000006-source/devenv.nix':
… from call site
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/modules.nix:825:137:
824| defs' = concatMap (m:
825| map (value: { inherit (m) file; inherit value; }) (builtins.addErrorContext "while evaluating definitions from `${m.file}':" (dischargeProperties m.value))
| ^
826| ) defs;
… while calling 'dischargeProperties'
at «github:NixOS/nixpkgs/2748d22b45a99fb2deafa5f11c7531c212b2cefa»/lib/modules.nix:896:25:
895| */
896| dischargeProperties = def:
| ^
897| if def._type or "" == "merge" then
… from call site
at «github:shazow/foundry.nix/ece7c960a440c6725a7a5576d1f49a5fabde3747»/flake.nix:18:23:
17| pkgs = import nixpkgs { inherit system; };
18| foundry-bin = import ./foundry-bin { inherit pkgs; };
| ^
19| # TODO: Add a source-based derivation someday
… while calling anonymous lambda
at «github:shazow/foundry.nix/ece7c960a440c6725a7a5576d1f49a5fabde3747»/foundry-bin/default.nix:1:1:
1| {pkgs ? import <nixpkgs> {}}: let
| ^
2| inherit (pkgs) stdenv lib;
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:81:25:
80| inputs = builtins.mapAttrs
81| (inputName: inputSpec: allNodes.${resolveInput inputSpec})
| ^
82| (node.inputs or {});
… from call site
at «nix-internal»/call-flake.nix:81:36:
80| inputs = builtins.mapAttrs
81| (inputName: inputSpec: allNodes.${resolveInput inputSpec})
| ^
82| (node.inputs or {});
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:39:13:
38| builtins.mapAttrs
39| (key: node:
| ^
40| let
error:
error: opening file '/nix/store/0n86420hx4bydkpx8q7a4vqw46f5ihhp-source/.devenv.flake.nix': No such file or directory
✖ Command produced the following output:
✔ Building shell in 1.1s.
Error: × Command `/nix/store/16z8qb8j04nzahvxgms9wn9qba1fjh4z-nix-devenv-
│ 2.21.0pre20240315_c5bbf14/bin/nix --show-trace --extra-experimental-
│ features nix-command --extra-experimental-features flakes --option
│ warn-dirty false --option eval-cache false --keep-going --max-
│ jobs 3 print-dev-env --profile /home/chrism/projects/foundry-
│ devenv/.devenv/gc/shell --option extra-substituters https://
│ devenv.cachix.org --option extra-trusted-public-keys devenv.cachix.org-
│ 1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw= nixpkgs-python.cachix.org-
│ 1:hxjI7pFxTyuTHn2NkvWCrAUcNZLNS3ZAvfYNuYifcEU=` failed with with exit
│ code 1
The nix path its looking in is some version of nixpkgs.
[chrism@optinix:~/projects/foundry-devenv]$ ls /nix/store/0n86420hx4bydkpx8q7a4vqw46f5ihhp-source
CONTRIBUTING.md default.nix flake.nix maintainers pkgs
COPYING doc lib nixos README.md
Removing the dependency on foundry-bin
allows for a shell:
[chrism@optinix:~/projects/foundry-devenv]$ git diff
diff --git a/devenv.nix b/devenv.nix
index ccdddcb..69c034f 100644
--- a/devenv.nix
+++ b/devenv.nix
@@ -15,7 +15,7 @@ in {
# https://devenv.sh/packages/
packages = [
- pkgs.foundry-bin
+# pkgs.foundry-bin
pkgs.git
pkgs.jq
pkgs.zip
[chrism@optinix:~/projects/foundry-devenv]$ devenv shell
• Building shell ...
• Using Cachix: devenv
✔ Building shell in 10.8s.
• Entering shell
✨ devenv 1.0.3 is out of date. Please update to 1.0.4: https://devenv.sh/getting-started/#installation
*********************************************************
* *
* Welcome to the Nix dev environment for foundry! *
* *
* For more information, please visit: *
* https://github.com/shazow/foundry.nix *
* https://github.com/cachix/devenv *
* *
*********************************************************
(devenv)
foundry-bin
is available only as a result of the following overlay:
inputs:
...
foundry-overlay:
url: path:./foundry-overlay
overlays:
- default
{
inputs = {
foundry.url = "github:shazow/foundry.nix/monthly";
};
outputs = { self, foundry, ... }: {
overlays.default = self: super: {
foundry-bin = foundry.defaultPackage.${self.system};
};
};
}
Does not appear to be related to being an overlay, because if I change things around such that we just use it directly:
inputs:
...
foundry:
url: github:shazow/foundry.nix/monthly
and
{ inputs, pkgs, ... }:
{
....
packages = [
inputs.foundry.defaultPackage.${pkgs.system}
...
]
}
We wind up with more or less the same traceback.
The simplest possible repro of the issue is this (my foundry-devenv repo fork "simplify-bug-repro" branch has a devenv.nix that incudes only the remote flake as a package):
[chrism@optinix:~/projects]$ git clone git@github.com:mcdonc/foundry-devenv.git
Cloning into 'foundry-devenv'...
remote: Enumerating objects: 39, done.
remote: Counting objects: 100% (39/39), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 39 (delta 16), reused 34 (delta 14), pack-reused 0
Receiving objects: 100% (39/39), 14.05 KiB | 1.28 MiB/s, done.
Resolving deltas: 100% (16/16), done.
[chrism@optinix:~/projects]$ cd foundry-devenv/
[chrism@optinix:~/projects/foundry-devenv]$ git checkout simplify-bug-repro
branch 'simplify-bug-repro' set up to track 'origin/simplify-bug-repro'.
Switched to a new branch 'simplify-bug-repro'
[chrism@optinix:~/projects/foundry-devenv]$ devenv shell
• Building shell ...
• Using Cachix: devenv
error:
… while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:9:12:
...
38| builtins.mapAttrs
39| (key: node:
| ^
40| let
error:
error: opening file '/nix/store/0n86420hx4bydkpx8q7a4vqw46f5ihhp-source/.devenv.flake.nix': No such file or directory
✖ Command produced the following output:
✔ Building shell in 1.1s.
Error: × Command `/nix/store/16z8qb8j04nzahvxgms9wn9qba1fjh4z-nix-devenv-
│ 2.21.0pre20240315_c5bbf14/bin/nix --show-trace --extra-experimental-
│ features nix-command --extra-experimental-features flakes --option
│ warn-dirty false --option eval-cache false --keep-going --max-jobs 3
│ print-dev-env --profile /home/chrism/projects/foundry-devenv/.devenv/
│ gc/shell --option extra-substituters https://devenv.cachix.org
│ --option extra-trusted-public-keys nixpkgs-python.cachix.org-
│ 1:hxjI7pFxTyuTHn2NkvWCrAUcNZLNS3ZAvfYNuYifcEU= devenv.cachix.org-
│ 1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=` failed with with exit
│ code 1
When I attempt to put the exact same files that are currently in the simplify-bug-repro branch into a subdir of examples
in devenv, devenv shell
runs fine against them and produces a shell without the exception. So that's a clue. :)
This is the same issue that I ran into and mentioned to Domen on Matrix a few days ago. I've created a barebones repo to demonstrate/reproduce the issue here, which (optionally) depends on a very simple flake which can be found here.
This issue came up for me when I started work on a pre-commit hook that scans repos for secrets using TruffleHog, which at the time I started was broken on Darwin in Nixpkgs. I noticed issues with pulling in my fixed version of the TruffleHog package:
I've noticed a few workarounds in this:
flake: false
is set in devenv.yaml
for that input.I've also noticed that the pre-commit hook I've defined actually works, even when devShell evaluation fails. That part evaluates fine.
I don't know how indicative it is, but the failure to find a .devenv.flake.nix
file in my case seems to happen in one of devenv's own dependencies, namely flake-parts
:
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:39:13:
38| builtins.mapAttrs
39| (key: node:
| ^
40| let
error:
error: opening file '/nix/store/2hc9lg18zd6yabw9jqj0wy3s9kyvkzp0-source/.devenv.flake.nix': No such file or directory
✖ Command produced the following output:
Error: × Command `/nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix --show-trace --extra-experimental-features nix-command --extra-experimental-features flakes
│ --option warn-dirty false --option eval-cache false --keep-going --max-jobs 5 print-dev-env --profile /Users/patcal04/Sandbox/empty-project/.devenv/gc/shell --option extra-
│ substituters https://devenv.cachix.org --option extra-trusted-public-keys devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=` failed with with exit code 1
direnv: failed to build the devenv environment. devenv.nix may contain errors. see above.
And when I inspect that Nix store path, I can see it's flake-parts
by the README:
# Flake Parts
_Core of a distributed framework for writing Nix Flakes._
`flake-parts` provides the options that represent standard flake attributes
and establishes a way of working with `system`.
Opinionated features are provided by an ecosystem of modules that you can import.
`flake-parts` _itself_ has the goal to be a minimal mirror of the Nix flake schema.```
The nix path its looking in is some version of nixpkgs.
[chrism@optinix:~/projects/foundry-devenv]$ ls /nix/store/0n86420hx4bydkpx8q7a4vqw46f5ihhp-source CONTRIBUTING.md default.nix flake.nix maintainers pkgs COPYING doc lib nixos README.md
It seems this varies. I've seen devenv mistakenly look for a .devenv.flake.nix
in not just nixpkgs, but also flake-parts. I think maybe we're looking for a .devenv.flake.nix
when we should still be looking for a flake.nix
.
I suspect the most fruitful place to look next will be comparing the .devenv.flake.nix
files generated by 0.x versions of devenv to those generated by 1.x versions, since that's where the shell is defined. Incidentally, the one I get with my example repo is this:
{
inputs =
let
version = "1.0.4";
system = "aarch64-darwin";
devenv_root = "/Users/patcal04/Sandbox/empty-project";
devenv_dotfile = ./.devenv;
devenv_dotfile_string = ".devenv";
container_name = null;
devenv_tmpdir = "/var/folders/fw/73494rb94f7b5vppp_c7_c5wl2kfgf/T/";
devenv_runtime = "/var/folders/fw/73494rb94f7b5vppp_c7_c5wl2kfgf/T/devenv-50fa3f4";
in {
pre-commit-hooks.url = "github:cachix/pre-commit-hooks.nix";
pre-commit-hooks.inputs.nixpkgs.follows = "nixpkgs";
nixpkgs.url = "github:cachix/devenv-nixpkgs/rolling";
devenv.url = "github:cachix/devenv?dir=src/modules";
} // (if builtins.pathExists (devenv_dotfile + "/flake.json")
then builtins.fromJSON (builtins.readFile (devenv_dotfile + "/flake.json"))
else { });
outputs = { nixpkgs, ... }@inputs:
let
version = "1.0.4";
system = "aarch64-darwin";
devenv_root = "/Users/patcal04/Sandbox/empty-project";
devenv_dotfile = ./.devenv;
devenv_dotfile_string = ".devenv";
container_name = null;
devenv_tmpdir = "/var/folders/fw/73494rb94f7b5vppp_c7_c5wl2kfgf/T/";
devenv_runtime = "/var/folders/fw/73494rb94f7b5vppp_c7_c5wl2kfgf/T/devenv-50fa3f4";
devenv =
if builtins.pathExists (devenv_dotfile + "/devenv.json")
then builtins.fromJSON (builtins.readFile (devenv_dotfile + "/devenv.json"))
else { };
getOverlays = inputName: inputAttrs:
map
(overlay:
let
input = inputs.${inputName} or (throw "No such input `${inputName}` while trying to configure overlays.");
in
input.overlays.${overlay} or (throw "Input `${inputName}` has no overlay called `${overlay}`. Supported overlays: ${nixpkgs.lib.concatStringsSep ", " (builtins.attrNames input.overlays)}"))
inputAttrs.overlays or [ ];
overlays = nixpkgs.lib.flatten (nixpkgs.lib.mapAttrsToList getOverlays (devenv.inputs or { }));
pkgs = import nixpkgs {
inherit system;
config = {
allowUnfree = devenv.allowUnfree or false;
allowBroken = devenv.allowBroken or false;
permittedInsecurePackages = devenv.permittedInsecurePackages or [ ];
};
inherit overlays;
};
lib = pkgs.lib;
importModule = path:
if lib.hasPrefix "./" path
then if lib.hasSuffix ".nix" path
then ./. + (builtins.substring 1 255 path)
else ./. + (builtins.substring 1 255 path) + "/devenv.nix"
else if lib.hasPrefix "../" path
then throw "devenv: ../ is not supported for imports"
else
let
paths = lib.splitString "/" path;
name = builtins.head paths;
input = inputs.${name} or (throw "Unknown input ${name}");
subpath = "/${lib.concatStringsSep "/" (builtins.tail paths)}";
devenvpath = "${input}" + subpath;
devenvdefaultpath = devenvpath + "/devenv.nix";
in
if lib.hasSuffix ".nix" devenvpath
then devenvpath
else if builtins.pathExists devenvdefaultpath
then devenvdefaultpath
else throw (devenvdefaultpath + " file does not exist for input ${name}.");
project = pkgs.lib.evalModules {
specialArgs = inputs // { inherit inputs pkgs; };
modules = [
(inputs.devenv.modules + /top-level.nix)
{
devenv.cliVersion = version;
devenv.root = devenv_root;
devenv.dotfile = devenv_root + "/" + devenv_dotfile_string;
}
(pkgs.lib.optionalAttrs (inputs.devenv.isTmpDir or false) {
devenv.tmpdir = devenv_tmpdir;
devenv.runtime = devenv_runtime;
})
(pkgs.lib.optionalAttrs (container_name != null) {
container.isBuilding = pkgs.lib.mkForce true;
containers.${container_name}.isBuilding = true;
})
] ++ (map importModule (devenv.imports or [ ])) ++ [
./devenv.nix
(devenv.devenv or { })
(if builtins.pathExists ./devenv.local.nix then ./devenv.local.nix else { })
];
};
config = project.config;
options = pkgs.nixosOptionsDoc {
options = builtins.removeAttrs project.options [ "_module" ];
# Unpack Nix types, e.g. literalExpression, mDoc.
transformOptions =
let isDocType = v: builtins.elem v [ "literalDocBook" "literalExpression" "literalMD" "mdDoc" ];
in lib.attrsets.mapAttrs (_: v:
if v ? _type && isDocType v._type then
v.text
else if v ? _type && v._type == "derivation" then
v.name
else
v
);
};
in
{
packages."${system}" = {
optionsJSON = options.optionsJSON;
inherit (config) info procfileScript procfileEnv procfile;
ci = config.ciDerivation;
};
devenv = config;
devShell."${system}" = config.shell;
};
}
It seems this varies. I've seen devenv mistakenly look for a
.devenv.flake.nix
in not just nixpkgs, but also flake-parts. I think maybe we're looking for a.devenv.flake.nix
when we should still be looking for aflake.nix
.
That sounds likely, thanks for relating it!
There must be some implicit thing that is consuming .devenv.flake.nix
because it's unreferenced in devenv's source other than creating it. Apologies if this is a FAQ but is there some feature of Nix itself that defers to things with some naming convention like this?
There must be some implicit thing that is consuming
.devenv.flake.nix
because it's unreferenced in devenv's source other than creating it. Apologies if this is a FAQ but is there some feature of Nix itself that defers to things with some naming convention like this?
Yes. I believe that's in Nix itself. Notice that the Nix path in our stack traces seems to be a devenv-specific fork. On my system, when I try to use nix repl
in a devenv project, I get different results with the system Nix than with the one that appears in my stack traces. On my work machine (which uses Nix 2.21.2):
empty-project on main [!⇡] on ☁️ (us-west-2)
❯ nix repl .
Nix 2.21.2
Type :? for help.
warning: Git tree '/Users/patcal04/Sandbox/empty-project' is dirty
error: path '/nix/store/0ibm4njlhjdxmikwjpy5zzrh99wxnnlv-source/.devenv.flake.nix' does not exist
empty-project on main [!⇡] on ☁️ (us-west-2)
❯ /nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix repl .
Nix 2.21.0
Type :? for help.
warning: Git tree '/Users/patcal04/Sandbox/empty-project' is dirty
warning: will not write lock file of flake 'git+file:///Users/patcal04/Sandbox/empty-project' because it has an unlocked input ('path:/Users/patcal04/Sandbox/nixpkgs')
Loading installable 'git+file:///Users/patcal04/Sandbox/empty-project#'...
Added 3 variables.
nix-repl> :help
The following commands are available:
<expr> Evaluate and print expression
<x> = <expr> Bind expression to variable
:a, :add <expr> Add attributes from resulting set to scope
:b <expr> Build a derivation
:bl <expr> Build a derivation, creating GC roots in the
working directory
:e, :edit <expr> Open package or function in $EDITOR
:i <expr> Build derivation, then install result into
current profile
:l, :load <path> Load Nix expression and add it to scope
:lf, :load-flake <ref> Load Nix flake and add it to scope
:p, :print <expr> Evaluate and print expression recursively
:q, :quit Exit nix-repl
:r, :reload Reload all files
:sh <expr> Build dependencies of derivation, then start
nix-shell
:t <expr> Describe result of evaluation
:u <expr> Build derivation, then start nix-shell
:doc <expr> Show documentation of a builtin function
:log <expr> Show logs for a derivation
:te, :trace-enable [bool] Enable, disable or toggle showing traces for
errors
:?, :help Brings up this help menu
nix-repl>
On my other machine, which I just did some testing on with my repos for reproduction this morning, I found a few other surprising things. (Post to follow, sorry in advance for double-posting.)
Check this out: 'stable' Nix (2.18.2) doesn't know how to open a flake repl in a devenv repo, but the Nix whose store path ends in -devenv
that shows up in my stack traces does:
devenv-problem-example on main [!⇡]
❯ nix repl .
Welcome to Nix 2.18.2. Type :? for help.
path '/Users/pxc/Documents/Code/devenv-problem-example' does not contain a 'flake.nix', searching up
error: path '/Users/pxc/Documents/Code/devenv-problem-example' is not part of a flake (neither it nor its parent directories contain a 'flake.nix' file)
devenv-problem-example on main [!⇡]
❯ /nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix repl .
Nix 2.21.0
Type :? for help.
warning: Git tree '/Users/pxc/Documents/Code/devenv-problem-example' is dirty
warning: will not write lock file of flake 'git+file:///Users/pxc/Documents/Code/devenv-problem-example' because it has an unlocked input ('path:/Users/pxc/Documents/Code/nixpkgs')
Loading installable 'git+file:///Users/pxc/Documents/Code/devenv-problem-example#'...
Added 3 variables.
nix-repl> :q
As for where in Nix to look, our clue is of course right there in the stack traces:
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:81:25:
80| inputs = builtins.mapAttrs
81| (inputName: inputSpec: allNodes.${resolveInput inputSpec})
| ^
82| (node.inputs or {});
… from call site
at «nix-internal»/call-flake.nix:81:36:
80| inputs = builtins.mapAttrs
81| (inputName: inputSpec: allNodes.${resolveInput inputSpec})
| ^
82| (node.inputs or {});
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:39:13:
38| builtins.mapAttrs
39| (key: node:
| ^
That's this file (permalink). I assume devenv patches this file somewhere to have it look for .devenv.flake.nix
rather than just flake.nix
I assume there's also some other trickery, that devenv perhaps invokes call-flake.nix
manually somehow, because this (from the nix repl
in my immediately previous post):
error: path '/nix/store/0ibm4njlhjdxmikwjpy5zzrh99wxnnlv-source/.devenv.flake.nix' does not exist
is complaining about a missing .devenv.flake.nix
in my example project, where of course it's present. It's missing in the Nix store because (per the devenv .gitignore
defaults), it's excluded from version control and the normal mechanisms won't copy it in there.
At any rate, our issues are likely in the patched-up Nix that it seems devenv uses for evaluation, perhaps in the embedded Nix code for libexpr
.
At any rate, our issues are likely in the patched-up Nix that it seems devenv uses for evaluation, perhaps in the embedded Nix code for
libexpr
.
Oh geez, well-spotted.
I assume there' also some other trickery, that devenv perhaps invokes
call-flake.nix
manually somehow, because this (from thenix repl
in my immediately previous post):
error: path '/nix/store/0ibm4njlhjdxmikwjpy5zzrh99wxnnlv-source/.devenv.flake.nix' does not exist
is complaining about a missing
.devenv.flake.nix
in my example project, where of course it's present. It's missing in the Nix store because (per the devenv.gitignore
defaults), it's excluded from version control and the normal mechanisms won't copy it in there.
Yep, check it out: one of the changes in Domen's fork of Nix is to copy untracked files into the store for flake repos: https://github.com/domenkozar/nix/commit/138a98698dc76ea207c746f82db50eac70255755
Incidentally I think that tells us we might (also?) want to look at libfetchers. Here's a starting point for files of interest:
nix-devenv on HEAD (b24a931) via C v15.0.0-clang
❯ rg -F '.devenv.flake.nix'
src/libfetchers/git-utils.cc
311: if (pathExists(cwd / ".devenv.flake.nix")) {
312: info.files.insert(CanonPath((cwd / ".devenv.flake.nix").string()).removePrefix(CanonPath(path.string())).rel());
src/libexpr/flake/flakeref.cc
120: if (!allowMissing && !pathExists(path + "/.devenv.flake.nix")){
121: notice("path '%s' does not contain a '.devenv.flake.nix', searching up",path);
127: if (pathExists(path + "/.devenv.flake.nix")) {
131: throw Error("path '%s' is not part of a flake (neither it nor its parent directories contain a '.devenv.flake.nix' file)", path);
139: throw BadURL("could not find a .devenv.flake.nix file");
145: if (!allowMissing && !pathExists(path + "/.devenv.flake.nix"))
146: throw BadURL("path '%s' is not a flake (because it doesn't contain a '.devenv.flake.nix' file)", path);
src/libexpr/flake/call-flake.nix
73: file = if builtins.pathExists (outPath + "/.devenv.flake.nix")
74: then "/.devenv.flake.nix"
src/libexpr/flake/flake.cc
181: auto devEnvPath = rootDir / flakeDir / ".devenv.flake.nix";
Here are links to the relevant files on the relevant branch:
(It looks like the error message is one of the generic ones from libutil
or libstore
.)
I'm poking through those files now, but I'll soon have to switch gears for work so I may not be able to follow up today.
I suspect the issue is how Nix resolves paths, while .devenv.flake.nix
is a red herring as it just looks it up and fallbacks to flake.nix
Testing a fix in #1137
As you've likely seen, this fix doesn't work because call-flake.nix
still needs to look for devenv.nix
at first, for print-dev-env
to work in the devenv project.
Could there be a bug in builtins.pathExists
that makes it fail for second-order dependencies when checking if .devenv.flake.nix
exists? What else could this issue be?
Here's the relevant bit of my stack trace when I tested this fix:
❯ /nix/store/hdlb4zssxmmc5qa8926kqmp66pxxv757-devenv-1.0.4/bin/devenv shell
• Building shell ...
error:
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:12:1:
11| # unlocked trees.
12| overrides:
| ^
13|
… from call site
at «nix-internal»/call-flake.nix:106:4:
105|
106| in allNodes.${lockFile.root}
| ^
107|
… while calling anonymous lambda
at «nix-internal»/call-flake.nix:39:13:
38| builtins.mapAttrs
39| (key: node:
| ^
40| let
error:
error: path '/flake.nix' does not exist in Git repository '/Users/pxc/Documents/Code/devenv-problem-example'
✖ Command produced the following output:
✔ Building shell in 0.1s.
Error: × Command `/nix/store/1zavp7qj3wic8b4vjnmsqbyg5dlpibn5-nix-devenv-2.21.0pre20240425_7c2a738/bin/nix --show-trace --extra-experimental-features nix-command --extra-experimental-
│ features flakes --option warn-dirty false --option eval-cache false --keep-going --max-jobs 5 print-dev-env --profile /Users/pxc/Documents/Code/devenv-problem-example/.devenv/gc/
│ shell` failed with with exit code 1
The bug is in pathExists
and how it handles virtual paths. I'm trying to come up with a way to avoid doing the conditional in Nix code, but rather in C++.
Thanks for the update. I just noticed from a bit of playing around. You've probably seen it, but it might be helpful for posterity, or linking to the upstream Nix issue eventually, if it hasn't been filed yet:
devenv-problem-example on main [!?⇡]
❯ /nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix eval --expr 'builtins.pathExists /nix/store/xvyy5vh6cg7958a26p2bqyz6jg5wkz4g-source/flake.nix'
false
devenv-problem-example on main [!?⇡]
❯ /nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix eval --impure --expr 'builtins.pathExists /nix/store/xvyy5vh6cg7958a26p2bqyz6jg5wkz4g-source/flake.nix'
true
devenv-problem-example on main [!?⇡]
❯ /nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix repl
Nix 2.21.0
Type :? for help.
nix-repl> builtins.pathExists /nix/store/xvyy5vh6cg7958a26p2bqyz6jg5wkz4g-source/flake.nix
true
pathExists
seems to behave differently in the repl from outside the repl (perhaps because the repl is always implicitly an impure environment?), and also when --impure
is passed vs. not. Perhaps even when flake evaluation is called with --impure
, the impurity applies only to the top-level flake?
(I'm also a bit surprised that any reference to something inside the Nix store is forbidden even during 'pure' evaluation, and that pathExists
simply returns false in that case rather than throwing an error. But I guess that's expected.)
This is mostly a side effect of https://github.com/NixOS/nix/pull/6530, which tries to abstract away flake paths.
It should be reported there if we want it to be fixed upstream.
Any word on this? This is a pretty killer bug for me. Should I just revert to older devenv
for the time being?
This is on the top of my priority list of next week.
Hello, we got this error in our GitHub Actions (following https://devenv.sh/integrations/github-actions/#run-multiple-commands) since yesterday. Would you know how to revert to the previous version in the GitHub Actions?
Looks like this builtins.pathExists
bug only affects flake inputs of type path
, or at least it doesn't affect flake inputs of type github
. By removing flake references of type path
(paths that, incidentally, are already in /nix/store
), I got my example repo to stop exhibiting the issue.
The repo my stuff was choking on was flake-parts
. When I added it to my inputs and got an appropriate change in my lock file, this issue went away.
I have the same issue with git+ssh://
inputs. Adding flake: false
works.
I have a similar issue, but mine seems unrelated? As I get a bit different message and using flake: false
doesn't help.
Describe the bug I had a working devenv with an overlay for foundry, but it started failing once I did
devenv update
(from 0.6.3 to 1.0.3).The process fails due to an error related to missing files and possible path issues. The specific error message indicates that the
.devenv.flake.nix
file cannot be found (despite it being there), leading to a failure in loading the environment. Here's the specific part of the error log that highlights the issue:To reproduce
Please find the necessary files (
devenv.nix
,devenv.yaml
, anddevenv.lock
) and the complete error logs in this https://gist.github.com/gonzaloetjo/41b62b2a97387b8b8029ba3c2fdd5d8c.Expected behavior The environment should load without errors, sourcing all necessary Nix packages and configurations. Previously, this setup loaded correctly without any issues.
Version
Previously, this configuration was working with version
0.6.3
.The repository for the current environment setup is found here.