Open edolstra opened 3 years ago
Another use case is https://github.com/NixOS/nix/issues/3946
So the fully general solution would be that buildPaths
is no longer a Store
method, but a static function that takes a stores configuration like this:
struct Stores {
// The store where the build result should appear.
ref<Store> destStore;
// The store that contains the .drv file to be built.
ref<Store> evalStore;
// The stores that can build stuff. (This replaces the `build-remote` hook.)
std::vector<ref<Store>> buildStores;
}
If something like --build-store
is ever implemented, couldn't that entirely replace the concept of remote builders too? It would be great to not have the two closely related but separate concepts of "remote builders" and "remote stores". So instead of having --builders
you could maybe just add multiple --build-store
arguments?
And while I'm here wishing for stuff, I could just as well throw in something like --fetch-store
to replace substituters too. I think it would be awesome to "trickle up" the internal store abstraction like this, since it makes the concepts simpler for the user, and at the same time more flexible which opens up for new interesting usage scenarios in distributing builds, caches and deployments.
@rickynils
When @edolstra writes
(This replaces the
build-remote
hook.)
This means it is doing exactly what you want!
Substitution and remote building in fact does use the store abstraction already. Hooray! The fact that remote building first shells to the build-remote
exe which then talks to the actual remote builder in the usual Store
interface way is just a historical artifact of the old perl days. This is what the above would do away with.
(In fact, we could perhaps have a "store with hooked building" adapter for backwards compat if someone really wants it!)
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/peer-to-peer-binary-cache-rfc-working-group-poll/29568/43
For several use cases it would be useful to distinguish between the stores used to store .drv files, to build derivations, and to store the resulting closure.
For example:
nix path-info --store https://cache.nixos.org nixpkgs#hello
doesn't work because the evaluation ofnixpkgs#hello
tries to create .drv files incache.nixos.org
./nix/store
is a high-latency remote store (as with flox), then copying lots of little .drv files to the remote is slow. So it's better to evaluate locally and only copy .drv closures to the remote as needed and in a single round-trip.