I'd like to be able to run nix-index frequently without re-doing all ~50k HTTP requests.
Many changes to nixpkgs only cause a few packages to rebuild, so I think if you check an existing database for each store path before going out to the remote binary cache, you'll usually find most of the Hydra-built contents are already present locally.
I think nix-index should always do this if the target database already exists. I'd like it if there were also an option to read from a different database than the one it's going to write to, analogous to git clone --reference.
It might also be worth recording the set of store paths which were not available from Hydra, but that isn't as reliable since Hydra might just not have gotten around to building some derivation until after the previous nix-index run. Offer this gated by a command-line option, perhaps?
I'd like to be able to run
nix-index
frequently without re-doing all ~50k HTTP requests.Many changes to nixpkgs only cause a few packages to rebuild, so I think if you check an existing database for each store path before going out to the remote binary cache, you'll usually find most of the Hydra-built contents are already present locally.
I think
nix-index
should always do this if the target database already exists. I'd like it if there were also an option to read from a different database than the one it's going to write to, analogous togit clone --reference
.It might also be worth recording the set of store paths which were not available from Hydra, but that isn't as reliable since Hydra might just not have gotten around to building some derivation until after the previous
nix-index
run. Offer this gated by a command-line option, perhaps?