Allow for automatic content discovery for cross-mounting blobs

[sargun]

When uploading to multiple registries, the user may or may not what other repositories exist in these registries. Therefore, a client may perform an unnecessary upload when the registry already has a given blob. This an optimization that allows the registry to perform the authz check and check if it can find the blob with a given the passed digest in its blobstore. If that blob is accessible (from an authz perspective) to the user, it can then perform the mount automatically on its behalf.

Because there is a potential a timing attack that could be used to disclose knowledge of whether or not the registry has a given blob (for example, a vulnerable version of a Linux image), this an optional feature for registries to implement.

Signed-off-by: Sargun Dhillon sargun@sargun.me

[sargun]

This follows up on the discussion in https://github.com/opencontainers/distribution-spec/issues/236.

[sargun]

Okay, this has like 80% of the work required: https://github.com/sargun/distribution/tree/cross-repo-mount

The configurations aren't user exposed, but the inner guts are wired up.

[sargun]

I changed the proposal / spec ever so slightly, and removed the sentence that said even if the blob wasn't found in the from repo, it could cross-mount from another repo. This turns out to be a pretty big headache.

[jonjohnsonjr]

removed the sentence that said even if the blob wasn't found in the from repo, it could cross-mount from another repo. This turns out to be a pretty big headache.

How is this semantically different?

[sargun]

@jonjohnsonjr If there are many repos with the same content, and the registry has the concept of "linking" inside of it, and it exposes this information in any way (which, as it turns out, the "reference" docker distribution does), it's confusing.

[jonjohnsonjr]

which, as it turns out, the "reference" docker distribution does

I'd be surprised that mounting a blob would interact with references, but also I don't think that I would predicate this PR on its interactions with a fork of distribution.

[sargun]

er, I don't mean "Reference" as in manifest references, I mean, the Docker Distribution implementation being the "reference implementation".

[sargun]

I added two FAQ entries explaining the situation slightly. Once this gets merged, I'll add a conformance test to make sure that old registries properly handle the behaviour that when from is omitted they still try to start the upload.

[sargun]

@jonjohnsonjr Updated based on your feedback

[sargun]

Issues:

https://github.com/opencontainers/distribution-spec/issues/281 https://github.com/opencontainers/distribution-spec/issues/282

Need someone to set the milestones.