Build: use `nix path-info` to test for build output availability

NixOS / hydra

Hydra, the Nix-based continuous build system

http://nixos.org/hydra

GNU General Public License v3.0

1.1k stars 291 forks source link

Build: use `nix path-info` to test for build output availability #1352

Open Ma27 opened 5 months ago

Ma27 commented 5 months ago

cc @dasj @Ericson2314

Hacky workaround for https://github.com/NixOS/nix/issues/9859, but after I managed to fix the issue on my end, I decided I could still submit it against upstream.

When using the Perl bindings, this usually takes the wrong store, i.e. auto even if store_uri is a flat-file binary cache.

Using fork/exec here on purpose to make sure that the output of nix path-info isn't spammed into stderr. This could probably done with system("foo &>/dev/null"), but I didn't want to worry about shell injection issues then. And finally, I can't seem to silence STDERR for system with a list as argument. Not a Perl dev though, so not sure if there's a better way.

Ericson2314 commented 5 months ago

I don't quite understand the things you are saying in that issue, but I soon as well just add the missing functionality. We have a nix-next branch so we can try it out right away.

lheckemann commented 5 months ago

Does this even work when using the nixos module? hydra-server.service doesn't seem to have nix on its PATH. I also wonder if this is an appropriate approach given the other ways in which the nix command could fail (experimental-features = nix-command missing?) and with no distinction drawn between that and the path not being valid.

Ma27 commented 5 months ago

It definitely works since I just realized I forgot to silence stdout and now I get paths printed to hydra-server's journal whenever I open a build view. ~~Not sure why yet :upside_down_face:~~

Nix is part of hydra-server's env:

hydra root ~ # cat /nix/store/d4w2gj8y1anymj9lsm3bih4v95f03bw3-hydra-0.1.20240126.1d1ec0b/bin/hydra-server|grep nix-
PATH=${PATH/':''/nix/store/7cibfivz81rpv8jhr53fj5hjlidpnykb-nix-2.19.3/bin'':'/':'}
PATH='/nix/store/7cibfivz81rpv8jhr53fj5hjlidpnykb-nix-2.19.3/bin'$PATH
export NIX_RELEASE='nix-2.19.3'

could fail (experimental-features = nix-command missing?)

Fair. Should probably use a nix-* equivalent then :shrug:

Ericson2314 commented 5 months ago

I am looking at the perl binding stuff.

Ericson2314 commented 5 months ago

I made https://github.com/NixOS/nix/pull/9863 to generalize the Perl bindings so we can fix this the right way. It would be nice to replace this with PR against nix-next using that branch.

I think we can basically do a big sed to update Hydra: it's fine to call openStore() again and again (without an argument, that is) because there is caching.