search

nix-community / nix-eval-jobs

Parallel nix evaluator with a streamable json output [maintainers @Mic92, @adisbladis]
GNU General Public License v3.0
140 stars 26 forks source link

`error: unexpected argument '3' Try '__build-remote --help' for more information.` #125

Open runeksvendsen opened 1 year ago

runeksvendsen commented 1 year ago

Running the command

sudo rm -rf $HOME/.tmp.1 $HOME/.tmp.2 ; nix-eval-jobs --gc-roots-dir $HOME/.tmp.1 --option max-jobs "0" --option builders "" --option store "local?root=$HOME/.tmp.2" --option substituters "" test/release.nix

where test/release.nix refers to https://github.com/dbornside-os-testorg/test/commit/759525f9a99f15d12d9df70087f78399f09fddc4

I get the following error:

error: unexpected argument '3'
Try '__build-remote --help' for more information.
error: unexpected EOF reading a line
{"error":"error: unexpected EOF reading a line"}

The same command works fine when run on a nix file that does not use IFD.

The reason I'm running nix-eval-jobs with these options is that I don't want it to build IFD derivations. Ie. I want it to fail in case of IFDs, but the above error looks like a bug, although I'm not sure what's going on.

Mic92 commented 1 year ago

Are you also using remote builder?

runeksvendsen commented 1 year ago

@Mic92 in practice, yes, we will be using a custom remote builder. However, adding e.g. the local nix daemon as a remote builder fails with the same error:

$ sudo rm -rf $HOME/.tmp.1 $HOME/.tmp.2 ; nix-eval-jobs --gc-roots-dir $HOME/.tmp.1 --option max-jobs "0" --option builders "daemon" --option store "local?root=$HOME/.tmp.2" --option substituters "" test/release.nix
error: unexpected argument '3'
Try '__build-remote --help' for more information.
error: unexpected EOF reading a line
{"error":"error: unexpected EOF reading a line"}
Mic92 commented 1 year ago

The nix library tries to self-execute, which we do not implement in nix-eval-jobs:

removing invalid path '/nix/store/5sgii519481kdsdg9kk027jlr7z641wd-bash-5.1.tar.gz'                                               
starting build hook '/nix/store/7ny84a740q0g0ghjrkyzy3cb8sqad3nh-nix-eval-jobs-0.0.1/bin/nix-eval-jobs __build-remote'
error: unexpected argument '7'                                                                                                    
Try '__build-remote --help' for more information.

Here is how nix handles this: https://github.com/NixOS/nix/blob/ac0fb38e8a5a25a84fa17704bd31b453211263eb/src/nix/main.cc#L270

I suspect hydra would also fail to evaluate this.

As of now --option max-jobs "0" is unsupported by nix-eval-jobs when using IFD due to this.

runeksvendsen commented 1 year ago

@Mic92

The nix library tries to self-execute, which we do not implement in nix-eval-jobs:

Why does it do that even though I pass both --option max-jobs "0" and --option builders ""? As far as I can see, nix-eval-jobs trying to spawn remote builders when builders is "" is a bug. Do you agree?
My expectation is that nix-eval-jobs should fail with the same message as nix-instantiate does when given the same options:

$ sudo rm -rf $HOME/.tmp.2 ; nix-instantiate --option max-jobs "0" --option builders "" --option store "local?root=$HOME/.tmp.2" --option substituters "" test/release.nix
error: unable to start any build; either increase '--max-jobs' or enable remote builds.
       https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html
(use '--show-trace' to show detailed location information)
runeksvendsen commented 1 year ago

Note: playing around with it some more, it appears that this error also occurs when only --option substituters "" and --option builders "" are present — ie. when I demand that everything be built locally.
So this command nix-eval-jobs command

sudo rm -rf $HOME/.tmp.1 $HOME/.tmp.2 ; nix-eval-jobs --gc-roots-dir $HOME/.tmp.1 --option store "local?root=$HOME/.tmp.2" --option builders "" --option substituters "" test/release.nix

fails with:

building '/nix/store/p6dnqlq09jwcka4acqyrxpja284yzlss-bash-5.1.tar.gz.drv'...
building '/nix/store/k31rqappkqncvgyhhski91cfn3zl6igy-bash51-001.drv'...
building '/nix/store/5fsjx4cx5chfiyv2shnnzqs2cl0bp3vq-bash51-002.drv'...
building '/nix/store/bnyq7bdgwj2w7nz98kc2xsy9vy09yfng-bash51-003.drv'...
building '/nix/store/6xa18rm747hsb6lmgknharv2ifzbk7n5-bash51-004.drv'...
building '/nix/store/1h2k4r9m23y3mkp8dzahfhf9izhcqy41-bash51-005.drv'...
building '/nix/store/gyhsjkaryybk7kgfbp6w9k9rymrxi82k-bash51-006.drv'...
building '/nix/store/q6kd3wqyg44fsnmap5v4vgwfljqa8qlj-bash51-007.drv'...
building '/nix/store/7i9060pvz2di4llhmjzb6fi6hkhxrs4y-bash51-008.drv'...
building '/nix/store/syqgms9jqlgbfqac4j4b0z2wjmhx4clm-bash51-009.drv'...
building '/nix/store/gzcxdb2nf7gxmy88zz88491la3ira8r3-bash51-010.drv'...
building '/nix/store/38n3s6l5ijf0hsqqbjbknj0ipwzya5vk-bash51-011.drv'...
building '/nix/store/p28nvi191giw15v888dyi4sn8ynczp88-bash51-012.drv'...
building '/nix/store/7qknalznr049gsw2g1q5asbz28al3ma2-bash51-013.drv'...
building '/nix/store/x7gk6bmimkbfz65ya3dvya2dnv4610kx-bash51-014.drv'...
building '/nix/store/ignk0x41n3d5w9dr4k5wbrgsj04ffgxc-bash51-015.drv'...
building '/nix/store/f19g1bx64frn83hsnvnq1s0x1yilhk18-Python-3.9.13.tar.xz.drv'...
building '/nix/store/x30m3rvz7j39imcm6i25mf444kiavlbp-acl-2.3.1.tar.gz.drv'...
building '/nix/store/hbd9bkhy0x7qfnl8p85793hh45yzdzsy-attr-2.5.1.tar.gz.drv'...
building '/nix/store/c99ihlhb2lh875spzsl6rnc4058grxvn-autoconf-2.71.tar.xz.drv'...
building '/nix/store/h92k47znafywj1vf1g5ck6x7q8lk8x28-autoconf-archive-2021.02.19.tar.xz.drv'...
building '/nix/store/3yar2pnvz7ll79z3jlzx09qnhrsi7zj5-automake-1.16.5.tar.xz.drv'...
building '/nix/store/9k07l1w51r9aakxxgqh2hnplmdhdm4mx-bash51-016.drv'...
building '/nix/store/is6j9vdyz6fanhrj5fva1hqmfywf0fh5-binutils-2.38.tar.bz2.drv'...
building '/nix/store/vgjfnqbxgxa8a5575bhq07nm35b2l31m-bison-3.8.2.tar.gz.drv'...
building '/nix/store/bzq60ip2z5xgi7jk6jgdw8cngfiwjrcm-bootstrap-tools.tar.xz.drv'...
building '/nix/store/b7irlwi2wjlx5aj1dghx4c8k3ax6m56q-busybox.drv'...
building '/nix/store/806qrynji7s8dayqd5gzqxk5s7816nyw-bzip2-1.0.6.0.2.tar.gz.drv'...
building '/nix/store/4dk9jfhw46cyk9c5n3vw8r6kcyp0gjgm-coreutils-9.0.tar.xz.drv'...
building '/nix/store/y3jr6xnfjncmgklis1lyrd08j9w46503-curl-7.83.1.tar.bz2.drv'...
building '/nix/store/0nb35wjnzrjkn0pxlazxnns9vv5l95c4-diffutils-3.8.tar.xz.drv'...
building '/nix/store/c4g14gya2arr3sqwwnjn76gizn4ks46h-ed-1.18.tar.lz.drv'...
building '/nix/store/0an3176n8cp5143c7l9dw1sq6vf90pcq-expat-2.4.8.tar.xz.drv'...
building '/nix/store/g4y4ssim94dkg5135xz7vj2i0f3jwgg6-file-5.41.tar.gz.drv'...
building '/nix/store/jxhwnvkpxrbl5rrf01zf3cf2p7v215pb-findutils-4.9.0.tar.xz.drv'...
building '/nix/store/xir6rxfd3wnq4lw8j3zksfkri490nzxp-gawk-5.1.1.tar.xz.drv'...
building '/nix/store/gp49lkiabq8dnx0ix9pam80n1p93p6q6-gcc-11.3.0.tar.xz.drv'...
building '/nix/store/34vpnpik7b1fb91gwv8pfw7s7v8yir01-gettext-0.21.tar.gz.drv'...
building '/nix/store/kf2lcp7fx5s8qsk7ls2ci8gp0m1nqnxj-gettext-1.07.tar.gz.drv'...
building '/nix/store/k3786wfzw637r7sylccdmm92saqp73d8-glibc-2.34.tar.xz.drv'...
building '/nix/store/l5zr5m1agvvnic49fg6qc44g5fgj3la1-glibc-reinstate-prlimit64-fallback.patch?id=eab07e78b691ae7866267fc04d31c7c3ad6b0eeb.drv'...
building '/nix/store/a5alapfd1s4rf9dbwb1h0mhlvanl3qvz-gmp-6.2.1.tar.bz2.drv'...
building '/nix/store/2bgknc6ism1p2wx0k88nz7g16i4zsqf0-grep-3.7.tar.xz.drv'...
building '/nix/store/6v6ld5igw5f9rw3gdc7w7s14cfpfq63c-gzip-1.12.tar.xz.drv'...
building '/nix/store/wsqm127z52159zzrd39ikvi07zcdk50d-help2man-1.49.1.tar.xz.drv'...
building '/nix/store/qxz8v1gxrrw9j138k2ajl36vcajra8mb-isl-0.20.tar.xz.drv'...
building '/nix/store/lkkzb8ybhp5jjgvb04b13n68i51lxwd9-keyutils-1.6.3.tar.gz.drv'...
building '/nix/store/0cixafavy49m0czb31sysr2w3wmc761j-krb5-1.19.3.tar.gz.drv'...
building '/nix/store/5s87b3i5xfdd1mrl2haavfjrzfp4w4zf-libelf-0.8.13.tar.gz.drv'...
building '/nix/store/fznarjiqmp7mvcgs5m5ssk38nn0k3bm4-libffi-3.4.2.tar.gz.drv'...
building '/nix/store/8cc7z3dqb1vzk76129is7iw1h8rz1qpg-libidn2-2.3.2.tar.gz.drv'...
building '/nix/store/vizgh9p9qc3v9f4c17f10smqb7zg90sg-libssh2-1.10.0.tar.gz.drv'...
building '/nix/store/rwvhsf6hna1j3sgw20f1b4gl9mp5w9vr-libtool-2.4.7.tar.gz.drv'...
building '/nix/store/navsvnff9yq84x03vmpxws02l8kh9ha5-libunistring-1.0.tar.gz.drv'...
building '/nix/store/n6hwl4y21nhvf6kdmnipdhphxd1nyq13-linux-5.17.tar.xz.drv'...
building '/nix/store/1zi0k7y01rhqr2gfqb42if0icswg65sj-locale-C.diff.drv'...
building '/nix/store/mxiibcd5b5v63fas7pfg2zavgp5bi5fk-lzip-1.23.tar.gz.drv'...
building '/nix/store/rr4cpsx1w84h2l1h71ch8spcp221kzkr-m4-1.4.19.tar.bz2.drv'...
building '/nix/store/jrxmi74200w7l24h0a6i99h2ck3f4r7m-make-4.3.tar.gz.drv'...
building '/nix/store/7r3b56dqf8jy4qd922i8ziisrn6ijikr-mpc-1.2.1.tar.gz.drv'...
building '/nix/store/lv1hwcmry75256awiysfa7mb93a0c49c-mpfr-4.1.0.tar.xz.drv'...
building '/nix/store/c9sgpp7j2skq276i7ws8xg96i0273jpk-nghttp2-1.47.0.tar.bz2.drv'...
building '/nix/store/7f7k8xda5nvvw4bjhzzdpkyk0kzvb0b1-openssl-1.1.1q.tar.gz.drv'...
building '/nix/store/angvcbhbr5p8viv4zphsxxx5nl4zr1hi-patch-2.7.6.tar.xz.drv'...
building '/nix/store/bmjrxb250i7jhhzz8gfmrprnaj6h0zp5-patchelf-0.14.5.tar.bz2.drv'...
building '/nix/store/ka746a7jhv7ii7lg92yibbi5n89gwaih-pcre-8.45.tar.bz2.drv'...
building '/nix/store/srr6xmjy79z5cn06mr7sx51p4hz8sv58-perl-5.34.1.tar.gz.drv'...
building '/nix/store/6b29gjz7rj4mw0ch0vy2m6qrqipz2bbb-pkg-config-0.29.2.tar.gz.drv'...
building '/nix/store/45j86ggi8mlpfslcrgvjf7m6phia21fp-raw.drv'...
building '/nix/store/wlldapf5bg58kivw520ll5bw0fmlaid7-raw.drv'...
building '/nix/store/cnh3rigjsk9vfismiz6cjx3qf6g0pipz-sed-4.8.tar.xz.drv'...
building '/nix/store/8yhywccj73wdxv579rz5lk45dpjfgn2w-tar-1.34.tar.xz.drv'...
building '/nix/store/8x8fismhjkqwcl5qc2rq2zpm70sm9vv7-texinfo-6.8.tar.xz.drv'...
building '/nix/store/01n3wxxw29wj2pkjqimmmjzv7pihzmd7-which-2.21.tar.gz.drv'...
building '/nix/store/8mg6zjhc5fbms6nm3snd41galx63ijhw-xz-5.2.5.tar.bz2.drv'...
building '/nix/store/ara7ym168qgzfzi2an6g594361xxk0c9-zlib-1.2.12.tar.gz.drv'...
error: unexpected argument '3'
Try '__build-remote --help' for more information.
error: unexpected EOF reading a line
{"error":"error: unexpected EOF reading a line"}

It's unclear to me why nix-eval-jobs is unable to build https://github.com/dbornside-os-testorg/test/commit/759525f9a99f15d12d9df70087f78399f09fddc4 locally.
When I run nix-instantiate in the same way, ie.

sudo rm -rf $HOME/.tmp.2 ; nix-instantiate --option store "local?root=$HOME/.tmp.2" --option substituters "" --option builders "" test/release.nix

it starts building stuff, as I would expect.

runeksvendsen commented 1 year ago

This issue is solved. A couple of points:

  1. The error {"error":"error: unexpected EOF reading a line"} is irrelevant and should be ignored
  2. The error error: unexpected argument '3' Try '__build-remote --help' for more information. is basically the expected error; nix-instantiate just has a more helpful error (error: unable to start any build; either increase '--max-jobs' [...])
  3. To get a more sensible error, pass the option --option build-hook ""
    • after which the error message will be {"error":"error: 'build-hook' setting is empty"}
  4. Even when failing with {"error":"error: 'build-hook' setting is empty"} the exit code is 0. Not sure whether this is a bug.