nix-community / nix-eval-jobs

Parallel nix evaluator with a streamable json output [maintainers @Mic92, @adisbladis]
GNU General Public License v3.0
140 stars 26 forks source link

`--flake` not working as expected #271

Closed johnrichardrinehart closed 8 months ago

johnrichardrinehart commented 9 months ago

Description

I'm finding that the top-level pkgs attribute from nixpkgs isn't accessible through nix-eval-jobs like it is with nix:

$ nix --version
nix (Nix) 2.17.1
$ nix-eval-jobs --flake 'github:nixos/nixpkgs#pkgs.hello' --gc-roots-dir /tmp/throwaway
warning: unknown setting 'allowed-users'
warning: unknown setting 'trusted-users'
error: flake 'github:nixos/nixpkgs' does not provide attribute 'pkgs.hello'
error: worker error: error: flake 'github:nixos/nixpkgs' does not provide attribute 'pkgs.hello''
$ nix build 'github:nixos/nixpkgs#pkgs.hello' -o ./result-hello && echo $? && readlink -f ./result-hello 
0
/nix/store/sbldylj3clbkc0aqvjjzfa6slp4zdvlj-hello-2.12.1

Is this expected/known?

Thanks for this tool! It's great!

Mic92 commented 8 months ago

This is the wrong attribute path. Try: nix-eval-jobs --flake 'github:nixos/nixpkgs#legacyPackages.x86_64-linux.hello' --gc-roots-dir /tmp/throwaway. nix-eval-jobs unless nix build and co wants absolute attribute paths for everything. It will not add implicit prefixes.

johnrichardrinehart commented 8 months ago

Thanks for your response. Did you mean "nix-eval-jobs unlike nix build and co" instead of

nix-eval-jobs unless nix build and co

Is there a reason this feature is not something in the scope of nix-eval-jobs? What design/API restrictions are there on nix-eval-jobs?

Mic92 commented 8 months ago

I think this behavior is only consistent with the rest of the cli. nix build .#package will try to build nix build .#packages.x86_64-linux.package whereas nix eval .#package will only try to eval .#package.

johnrichardrinehart commented 8 months ago

Okay, that makes sense. Thanks for explaining. I'm still concerned there's a lack of parity between nix-eval-jobs and nix eval in the flake URI sense. Do you think the below should work for nix-eval-jobs? If not, then why?

$ nix-eval-jobs '.#ffmpeg.lib.outPath'
warning: unknown setting 'allowed-users'
warning: unknown setting 'trusted-users'
warning: `--gc-roots-dir' not specified
error: getting status of '/home/john/code/repos/others/nixos/nixpkgs/.#ffmpeg.lib.outPath': No such file or directory
error: worker error: error: getting status of '/home/john/code/repos/others/nixos/nixpkgs/.#ffmpeg.lib.outPath': No such file or directory
$ nix eval '.#ffmpeg.lib.outPath'     
"/nix/store/sqvg9xfw3qaqd54rk8q185zpzn7gkqgk-ffmpeg-5.1.3-lib"
Mic92 commented 8 months ago

First you have to use --flake for flakes. But in your example it's not a derivation, nix-eval-jobs won't work for string output. In that case just use nix eval directly. nix-eval-jobs is built for evaluating large sets of derivations.

johnrichardrinehart commented 8 months ago

Even if I revise it to use --flake with an attribute recognized by nix eval it still fails:

$ nix-eval-jobs --flake '.#ffmpeg.lib'
warning: unknown setting 'allowed-users'
warning: unknown setting 'trusted-users'
warning: `--gc-roots-dir' not specified
error: flake 'git+file:///home/john/code/repos/others/nixos/nixpkgs' does not provide attribute 'ffmpeg.lib'
error: worker error: error: flake 'git+file:///home/john/code/repos/others/nixos/nixpkgs' does not provide attribute 'ffmpeg.lib'

whereas

$ nix eval '.#ffmpeg.lib'
error: stack overflow (possible infinite recursion)

So, according to nix eval, ffmpeg.lib is an existing attribute. Also, the earlier command (obtaining the outPath) is a child of this attribute so the parent must exist.

So, is there agreement between nix-eval-jobs and nix eval about how to process flake arguments?

Mic92 commented 8 months ago

I don't know why nix eval goes into infinite recursion here, it's definitely not meant to be used on derivations like this. However nix-eval-jobs --flake '.#legacyPackages.x86_64-linux.ffmpeg.lib' works as expected.

johnrichardrinehart commented 8 months ago

I don't know why nix eval goes into infinite recursion here, it's definitely not meant to be used on derivations like this.

Yeah, it's always been a source of frustration for me that nix eval won't yield results for anything more complex than a bool or string or path. It'd be nice if nix eval on an attribute which is associated with a function or attribute set would dump a string representation of the function or attribute set.

However nix-eval-jobs --flake '.#legacyPackages.x86_64-linux.ffmpeg.lib' works as expected.

For sure. I guess my confusion is that nix eval seems to support some indirection to resolve the derivation to evaluate. However, nix-eval-jobs --flake doesn't support the same indirection. So, is it correct to say that users should understand that --flake can only act on roots visible through nix flake show (i.e. no magic)?

Mic92 commented 8 months ago

Yes. I consider nix-eval-jobs not very useful on its own, but it enables to build more complex tooling such CI integrations or nix build tools on top.