search

cachix / devenv

Fast, Declarative, Reproducible, and Composable Developer Environments
https://devenv.sh
Apache License 2.0
3.47k stars 252 forks source link

devenv test inconsistency in v1.0.4 #1149

Closed norbertwnuk closed 1 week ago

norbertwnuk commented 2 weeks ago

I found that v1.0.4 changed the way how test command works when the imports are used. The difference is visible even in the devenv examples (log below), the CI is the same time OK however it run those tests differently - link What is the nature of this difference?

~/src/GitHub/devenv/examples/imports$ nix --version
nix (Nix) 2.18.0
~/src/GitHub/devenv/examples/imports$ devenv version
devenv 1.0.4 (x86_64-linux)

~/src/GitHub/devenv/examples/imports$ devenv shell
• Building shell ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 2.8s.
• Entering shell

(devenv) ~/src/GitHub/devenv/examples/imports$ devenv test
• Overriding .devenv to .devenv.w39jD8pAld6S
• Building tests ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building tests in 3.4s.
warning: creating lock file '/examples/imports/devenv.lock'
• Running tests ...
• Building shell ...
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 3.6s.
• Setting up shell environment ...
• Testing ...
• Running .test.sh...
FILE=1
FOLDER=1
✔ Running tests in 5.0s.
• Tests passed :)

(devenv) ~/src/GitHub/devenv/examples/imports$ exit
exit

~/src/GitHub/devenv/examples/imports$ devenv test
• Overriding .devenv to .devenv.WLOCka79uPJu
• Building tests ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building tests in 2.3s.
warning: creating lock file '/examples/imports/devenv.lock'
• Running tests ...
• Building shell ...
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 3.0s.
• Setting up shell environment ...
• Testing ...
• Running .test.sh...
✔ Running tests in 4.2s.
✖ Tests failed :(
Error:   × Tests failed
domenkozar commented 2 weeks ago

What's the version of devenv you're comparing to?

norbertwnuk commented 2 weeks ago

v1.0.3 works as below:

~/src/GitHub/devenv/examples/imports$ nix-shell -p devenv -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/e89cf1c932006531f454de7d652163a9a5c86668.tar.gz
[nix-shell:~/src/GitHub/devenv/examples/imports]$ devenv version
devenv 1.0.3 (x86_64-linux)
[nix-shell:~/src/GitHub/devenv/examples/imports]$ devenv test   
• Overriding .devenv to .devenv.KBaU1KJRiYTK
• Building tests ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building tests in 5.9s.
warning: creating lock file '/examples/imports/devenv.lock'
• Running tests ...
• Building shell ...
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 3.3s.
✨ devenv 1.0.3 is out of date. Please update to 1.0.4: https://devenv.sh/getting-started/#installation
• Setting up shell environment ...
✨ devenv 1.0.3 is out of date. Please update to 1.0.4: https://devenv.sh/getting-started/#installation
• Testing ...
• Running .test.sh...
FILE=1
FOLDER=1
✔ Running tests in 5.2s.
• Tests passed :)
mcdonc commented 2 weeks ago

I can't reproduce this locally using a profile:

[chrism@optinix:~/projects/devenv/examples/imports]$ nix --version
nix (Nix) 2.18.1

[chrism@optinix:~/projects/devenv]$ git log --oneline|head -5
5684af4 v1.0.4
a4eb473 Fix #1086: fix gleam
45796a9 Fix #1118: add --offline
7ddfa99 Merge pull request #1125 from ri0t/patch-1
ed0ff45 Update home.html

[chrism@optinix:~/projects/devenv]$ nix profile install --accept-flake-config .

[chrism@optinix:~/projects/devenv]$ cd examples/imports/

[chrism@optinix:~/projects/devenv/examples/imports]$ rm -rf .devenv .devenv.flake.nix devenv.lock 

[chrism@optinix:~/projects/devenv/examples/imports]$ devenv shell
• Building shell ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 2.1s.
• Entering shell
(devenv) 
[chrism@optinix:~/projects/devenv/examples/imports]$ devenv test
• Overriding .devenv to .devenv.RSthmZc44B6H
• Building tests ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building tests in 1.3s.
warning: creating lock file '/examples/imports/devenv.lock'
• Running tests ...
• Building shell ...
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 2.5s.
• Setting up shell environment ...
• Testing ...
• Running .test.sh...
FILE=1
FOLDER=1
✔ Running tests in 3.0s.
• Tests passed :)
(devenv) 
[chrism@optinix:~/projects/devenv/examples/imports]$ exit
exit

[chrism@optinix:~/projects/devenv/examples/imports]$ devenv test
• Overriding .devenv to .devenv.SIIZpyFwUWZe
• Building tests ...
• Using Cachix: devenv
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building tests in 1.3s.
warning: creating lock file '/examples/imports/devenv.lock'
• Running tests ...
• Building shell ...
warning: creating lock file '/examples/imports/devenv.lock'
✔ Building shell in 2.3s.
• Setting up shell environment ...
• Testing ...
• Running .test.sh...
FILE=1
FOLDER=1
✔ Running tests in 2.7s.
• Tests passed :)
imcom commented 2 weeks ago

in my case, the trigger is the python.version

  languages.python.enable = true;
  languages.python.version = "3.11.8";

With this setting, devenv test fails

       … from call site
         at «github:cachix/devenv-nixpkgs/285676e87ad9f0ca23d8714a6ab61e7e027020c6»/lib/types.nix:263:25:
          262|                 file = def.file;
          263|                 value = def.value arg;
             |                         ^
          264|               }) defs);

       … while calling 'getInput'
         at «github:cachix/devenv/11e7d3ef56f542e70f138069fe9b95401c2143c8»/src/modules/lib.nix:11:16:
           10|   config.lib = {
           11|     getInput = { name, url, attribute, follows ? [ ] }:
             |                ^
           12|       let

       error:
       error: To use 'languages.python.version', run the following command:

         $ devenv inputs add nixpkgs-python github:cachix/nixpkgs-python --follows nixpkgs

✖ Command produced the following output:

✔ Building tests in 2.2s.
Error:   × Command `/nix/store/as7cgg7qvnkamrcycbgcdvhs0kv7wphx-nix-2.21-devenv/bin/nix --show-trace --extra-experimental-features nix-command --extra-experimental-features flakes
  │ --option warn-dirty false --option eval-cache false --keep-going --max-jobs 5 build .#devenv.test --no-link --print-out-paths --option extra-substituters https://nixpkgs-
  │ python.cachix.org https://devenv.cachix.org --option extra-trusted-public-keys devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw= nixpkgs-python.cachix.org-
  │ 1:hxjI7pFxTyuTHn2NkvWCrAUcNZLNS3ZAvfYNuYifcEU=` failed with with exit code 1
devenv 1.0.4 (aarch64-darwin)
nix (Nix) 2.20.5
mcdonc commented 2 weeks ago

in my case, the trigger is the python.version

Pretty sure this is a different issue than the one described by the OP. If applicable, can you open a new issue with a way to reproduce it?

imcom commented 2 weeks ago

in my case, the trigger is the python.version

Pretty sure this is a different issue than the one described by the OP. If applicable, can you open a new issue with a way to reproduce it?

sure, let me file a new one

imcom commented 2 weeks ago

@mcdonc fyi https://github.com/cachix/devenv/issues/1151 thx

norbertwnuk commented 2 weeks ago

I'm pretty sure that in my example the list of imports was just empty, since in the more complex example I seen packages missing that would normally be added by the flake import. In the example it was visible as a lack of ENV variables from imports... however the master version works OK (as shown by mcdonc) for both the example and my own project... apparently something was already fixed.

domenkozar commented 1 week ago

Can we close this then?

norbertwnuk commented 1 week ago

Yes, will test it again when v1.0.5 will be released.