Registry metadata support in application manifest

[vdice]

The topic of registry metadata support in the Spin application manifest has come up a few times, most recently via https://github.com/fermyon/spin/issues/2612 where it was mentioned that declaration of annotations in the manifest itself could be useful, in addition to or as defaults for those that may be specified via the CLI at time of spin registry push.

Initial brainstorm

The conversation in #2612 provoked a quick initial brainstorm that perhaps a top-level [registry] section could be a good way to go. In addition to a [registry.annotations] section, the section could include other relevant metadata, such as:

• name of registry, eg ghcr.io/vdice
• image (OCI image/artifact name, defaulting to application name but could be overridden here, eg myapp)
• tag (image tag, defaulting to application version, but could be overridden here, eg latest).

All together, the following would be an envisioned example:

[registry]
name = "ghcr.io/vdice"
image = "myapp"
version = "latest"
[registry.annotations]
"org.opencontainers.image.url" = "https://www.example.com"

(Note, annotation keys including .'s would need to be quoted.)

In this example, CLI support would be added such that spin registry push would just work, with no need to supply a <REFERENCE>. (Though, if/when supplied, it would take precedence.) Instead, the defaults declared in the manifest would be used. In this case, the default reference published would be ghcr.io/vdice/myapp:latest.

(TBD the CLI behavior when a full default reference cannot be derived from the manifest, for example if registry is missing but image and version are present, and when no <REFERENCE> is supplied at time of command invocation.)

Open questions

• First, is this general approach something we'd like to pursue? If so, a SIP might be a preferred first action item from this issue.

• What other metadata or sub-sections might we envision under [registry]?

• Web Assembly Component Registry interactions?

  How would this design interact with a Web Assembly component registry? All of the above assumes the underlying registry is an OCI registry and the resulting image/artifact is an opinionated (read: customized OCI manifest layout and push/pull protocol) Spin app. (Conveniently, for the former, it currently looks like wasm component registries also use OCI registries under the hood.) What if Spin application authors would also (or instead) like to publish just the wasm component to a dedicated wasm component registry? Ref https://github.com/fermyon/spin/issues/2536 and a bit more tangentially, https://github.com/fermyon/spin/issues/1182.

  My first hunch would be that all of the metadata supplied in [registry] would apply to the published wasm component(s) as well, unless somehow specified otherwise. Unless we think of any design-altering items now, perhaps we're okay to relegate the specifics to the implementor of #2536.

[itowlson]

This ia a great discussion - thank you for writing all this up.

A consideration here is what is part of the application definition and what is part of a particular environment (or a particular publish operation). For example, if you and I are collaborating on a project, I would be pushing my local builds to ghcr.io/itowlson and you would push yours to ghcr.io/vdice (and CI would push the releases to ghcr.io/fermyon).

Of course, I can override the default registry name on the spin registry push command line and be no worse off, because I would have had to do that anyway, and hey at least you get to save some keystrokes. And perhaps users would set the registry name only on apps not intended for collaboration. But this is the kind of thing we'll need to be thinking about if we put publishing info into the app definition itself.

version is an example of "is it part of the app definition of part of this publish operation?" Do I want to perpetually push to v0.1.0, or do I want to be prompted to think about it on each push? Again, there is an argument that if I want to be prompted I can just not put the field in. But again it's worth making sure we aren't equipping users with footguns in the name of convenience...!

[NissesSenap]

Just thinking out loud.

I understand it's early days for component packaging, but is there any standard annotation that exist today? The only thing I found when searching for annotation in the registry repo was this

For me, it sounds reasonable to have similar annotations, but I would assume they won't start with stuff like org.opencontainers, but there will be some other default domain.

This is a bit more related to how you would publish remote components, but if it is something like spin registry push --component it should be easy to set default annotations, depending on if it's a container or component based.

One potential toml syntax could be to make a clear destination already for component and container based artifacts

[registry.annotations]
"useful.annoation" = "for both component and oci"
[registry.oci]
name = "ghcr.io/vdice"
image = "myapp"
version = "latest"
[registry.oci.annotations]
"org.opencontainers.image.url" = "https://www.example.com"

[registry.warg.annotations]
" warg.image.url" = "https://www.example.com"

[NissesSenap]

This ia a great discussion - thank you for writing all this up.

A consideration here is what is part of the application definition and what is part of a particular environment (or a particular publish operation). For example, if you and I are collaborating on a project, I would be pushing my local builds to ghcr.io/itowlson and you would push yours to ghcr.io/vdice (and CI would push the releases to ghcr.io/fermyon).

Of course, I can override the default registry name on the spin registry push command line and be no worse off, because I would have had to do that anyway, and hey at least you get to save some keystrokes. And perhaps users would set the registry name only on apps not intended for collaboration. But this is the kind of thing we'll need to be thinking about if we put publishing info into the app definition itself.

For me, your point is mostly an issue in the open-source world (not to be neglected of course), but at least where I have worked we never had the developers pushing containers to their own repos, since it would share company IP outside the company domain. In general, I only have CI build my container images based on branches.

So for me, I would always have the default value of CI in the spin.toml file, and if a developer wants to push it somewhere else, they would notice rather quickly that they don't have access to the repo, and adjust accordingly. But as you say, one way is of course to just leave it blank, and have everyone just write registry in that repo.