Manually adding/fetching content-addressed data with references

[Ericson2314]

As @fricklerhandwerk and I discussed on a few occasions, I think Nix has too much lingering traces of a "sources vs builds" dichotomy: in particular, it there should not be artificial restrictions on the ways that content-addressed store objects are created.

If the content-addressed store object can be the result of the build, then it should be possible to also create that same store object my manually adding and by fetching. No exceptions.

I recently got us closer to this by making all ContentAddressMethods be supported on Store::addToStore and friends. However, reference are still a weak sport;

• self-references are especially poorly supported outside of builds
• non-self references are also not supported by fixed-output derivations
• builtin eval-time fetchers support neither of these (but maybe that's OK)
• CLI supports neither of these (though has commented-out code to support self-references)

This was a sort of low-priority missing feature for me until #9911 came around: IIUC @puckipedia has this exact use case and a perhaps-dubious bugix broke her work-around in lieu of Nix properly supporting this feature. If we're going to continue treating that change as a proper bug-fix, we better damn well support the feature properly.

[puckipedia]

This only covers CA objects (and not input-addressed), right?

[thufschmitt]

Related: https://github.com/NixOS/nix/issues/4859 (and https://github.com/NixOS/nix/issues/5509)

[Ericson2314]

@puckipedia Yes. It is less clear to me how one can safely manually create an input-addressed path.

[puckipedia]

I meant references on input-addressed paths; as i was somewhat (but possibly wrongly, due to issue 5509) under the impression CA paths couldn't refer to input addressed paths.

[Ericson2314]

Content-addressed store objects can in fact refer to non-CA store objects, because current references are always merely a set of store paths.

I'm not totally comfortable with that, but the way that would change would be with the creation of a new "deeply content-addressed" flavor of store object, which would have to proove the deepness by storing the references in a different format. The existing form would continue to exist.