NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
12.71k stars 1.51k forks source link

feature: `--target-store` #8100

Open nrdxp opened 1 year ago

nrdxp commented 1 year ago

Is your feature request related to a problem? Please describe.

There is a clear desire (links in context) to have some kind of cache awareness to the build step. In other words, like how Nix substitutes derivations if possible instead of building, what if all we want to build for is to popullate the cache? There is no point to fetch the result to our local machine if it already exists in the desired location in that case, and its just a waste of bandwidth.

Describe the solution you'd like A couple of proposals have come up each with their merits. This one simply builds off the direction the new cli has been taking with the --store and --eval-store flags, and suggests we also simply add a --target-store.

Then even if the derivation doesn't exist on --store, if it exists on the --target-store we skip the build. Additionally, once the build has finished, we can automatically copy the result to the target in a single step that is also clear to the reader semantically (imo at least).

You could have some auxilary settings to control for things like:

Describe alternatives you've considered I thought #7587 would be a decent solution, but its a bit more error prone than I would have liked and ultimately this information needs to parsed to decide whether or not to build anyway, so we might as well just incooperate that into the build command itself.

3946 might actually be a duplicate of this, but perhaps the proposed design here is a little different. There we are just simply looking to build package that are lacking substitutes to decide what to do with them at a later step, whereas here we incooperate the desired result directly into the command invocation (to copy the build result to some target). To me, its different enough to warrant its own issue.

Additional context I put together: https://github.com/divnix/nix-uncached

To try and address this problem, but we dediced to actually stop using it. It does not do exactly what we want for various reasons that I will not elaborate here, but essentially we just fall back to parsing nix-build --dry-run for now, which has its own set of difficulties, that I feel can only be addressed suitably by this proposal.

Ericson2314 commented 1 year ago

--target-store is the same as --store, but adding a layering mechanism. You want the layering mechanism separate from which store it is being used for.