search

NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
12.8k stars 1.52k forks source link

`nix-build` should try each binary cache in order #2715

Open vaibhavsagar opened 5 years ago

vaibhavsagar commented 5 years ago

I have >1 binary cache configured, and if any one of them is down my builds fail, e.g.

$ nix-build
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 335 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 571 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 1307 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 2771 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 4605 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 9823 ms
warning: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6); retrying in 21372 ms
error: unable to download 'https://pact.cachix.org/4jdq59q6s6gpkwr0rcjg309cj7dyckh8.narinfo': Couldn't resolve host name (6)
(use '--show-trace' to show detailed location information)
ERROR: Job failed: exit status 1

I think Nix should try each cache sequentially before failing.

matthewbauer commented 5 years ago

For reference, I think this was fixed in https://github.com/NixOS/nix/commit/bba3f0a308cceb56bad4aa1efe13927360ae463f

cumber commented 5 years ago

@matthewbauer That commit is labelled as being in nix 2.2.2, which I'm using, and I still encounter this problem. Even when passing --fallback (which I don't really want to use all the time anyway, for the same reason --fallback isn't the default; I'd like an error rather than fall back to build-from-source if all of the binary caches aren't available).

My use case is using machines nix machines on a local network as build caches for each other using nix-server. Sometimes some of them are powered off, or I take a laptop roaming to a different network. I'd like a configuration where if any of my machines are reachable and have the thing I'm building then I copy it from there, and otherwise transparently fall back to cache.nixos.org, without having to change my configuration or use different commands based on which caches I think are available. This seems to be impossible at the moment.

cumber commented 5 years ago

No I was mistaken; adding --fallback does works. It's still not great for my purposes, since it spends about 30 seconds retrying each cache before moving on to the next one.

What I'd really like is to be able to distinguish, in the configuration, between caches that are to be opportunistically used if available and ones that must be available (like cache.nixos.org). I'd then expect the --fallback to essentially treat all binary caches as opportunistic ones, allowing build-from-source even if the usually-required ones are offline.

The current implementation intention seems to be that all binary caches are required to be available, and I should use --fallback if I want to tolerate a rare unavailability. It's not great for caches when unavailability is common and expected.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

vaibhavsagar commented 3 years ago

This is still an issue.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info