proposal: Tag deletion/better tag API

[jamesorlakin]

This is really an attempt at ratifying this PR on the Docker distribution repo.

I confess I have little knowledge of the underlying workings (digests vs tags vs manifests, etc), but if this PR has been completed can't we evolve the spec to support tag deletion? If tags can be easily created, why not allow registries to support a standardised way of mutating/deleting them? Deleting by manifest seems a bit overkill and imprecise, and there's many an example of people being caught out by this behaviour as it can take other tags down with it. The workaround involving uploading a new image to an existing tag to get a uniqe manifest to delete seems a bit silly.

I get that some public regstries MAY not want this, so what's to say the spec can't account for it being an optional implementation? Tags are first-class citizens to most end-users really.

Thank you.

[amouat]

I guess the idea behind deleting by manifest is that it unambiguously deletes something - regardless of which tags point to the manifest, that manifest and all tags pointing to it will be removed. The delete blob and delete manifest can therefore be used to quickly remove data that shouldn't have been upload e.g. user data uploaded by accident.

I'm not saying that it should stay. Just that there was a reasonable use case. In reality the operation is so dangerous that I think most registries ignore it.

It seems reasonable to me to have a way to delete tags as well. Does anyone know why this is explicitly disallowed? The linked PR from distribution (with some crazy workarounds!) certainly makes it clear that there is user demand for deleting tags.

[jamesorlakin]

Very much agreed. I feel tags should be a bit like Git branches - merely pointers to a commit. These pointers can be changed at any time or deleted. The actual commit the branch points to isn't deleted until garbage collection.

I guess my simple question for maintainers is 'have you ever deleted a Git branch on a public repo?'. If yes, you can see why such a feature would be useful 😉

[amouat]

Thinking about it more, I guess the reason tag deletion wasn't added was because of the ambiguity as to whether deleting a tag removes any data or other tags. As per your proposal, it should explicitly not delete other tags and blobs would only be deleted when nothing references them. (I would hesitate to mention garbage collection, as that's implementation dependent).

[misuzu]

Related: https://gitlab.com/gitlab-org/container-registry/-/merge_requests/51

[jdolitsky]

I think it's reasonable to expect that you can delete things you create with the same API. I think this should be revisited after a 1.0.0 release

[sudo-bmitch]

In addition to a tag deletion API, I'd like to see the tag/list API return more than a list of strings. It would be useful to get a list of json including metadata, e.g.:

[
  {
    "name": "latest",
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:cbd301cb1beb0ab38bea2c7ab0e852963115ab33f82cdcbde820ee3d8cda7671"
  },
  {
    "name": "v1.1",
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:92788c54472c8f9e42c99d45a10fd62a1a4d5d40963d67eacc86e6e10cf3ce5f"
  }
]

This way calling the list API doesn't also have to call get or head on every tag to identify all the current digests within a repository.

[jonjohnsonjr]

GCR not only allows you to delete tags, but also requires you to delete by tag before you delete by digest, to avoid accidentally deleting something important. DockerHub just disabled the deletion API altogether for similar reasons, IIUC. Most registries don't allow deletion by tag at all, I believe.

A good place to start would be to enumerate all existing behavior and try to figure out the rationale (e.g. by asking the authors).

[joaodrp]

I understand the concern around race conditions during tag deletes, but shouldn't we leave that implementation concern to registry providers? At least Google, AWS, and GitLab already support deleting tags in their container registries, so it's feasible.

What's stopping us from defining a contract for this functionality?

[jonjohnsonjr]

Tag deletion was added: https://github.com/opencontainers/distribution-spec/blob/main/spec.md#deleting-tags

On Sat, May 22, 2021 at 10:40 AM João Pereira @.***> wrote:

I understand the concern around race conditions during tag deletes, but shouldn't we leave that implementation concern to registry providers? At least Google, AWS, and GitLab already support deleting tags in their container registries, so it's feasible.

What's stopping us from adding this functionality to the spec?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/opencontainers/distribution-spec/issues/114#issuecomment-846441097, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIJGZRN7FILD2LEPBR7AV3TO7UCTANCNFSM4LR3UB5A .

[joaodrp]

Tag deletion was added: https://github.com/opencontainers/distribution-spec/blob/main/spec.md#deleting-tags …

Great, thanks! Should this issue be closed then?

[jamesorlakin]

@joaodrp seeing as I originally opened this looking for tag deletion, I'm happy this is now a part of the spec and can be closed. I'll leave that to a maintainer though, it's been a while since I've been in the container ecosystem unfortunately.

From what I can piece v1.0.0 of this spec is rather new, so I'm assuming it may take registeries a litte while to catch up?

[jonjohnsonjr]

I think it's probably safe to close this.

There was some mention of a better listing API in this thread, which is captured here: https://github.com/opencontainers/distribution-spec/issues/222

We also have https://github.com/opencontainers/distribution-spec/issues/250 which gets into fun details around race conditions with tag mutation.