Add support for `org.opencontainers.image.revision` annotation

[carlcsaposs-canonical]

What needs to get done

Add option to rockcraft pack to specify org.opencontainers.image.revision annotation value or automatically use latest git commit hash

Why it needs to get done

So that it's possible to find git commit hash that a rock was built from

[syncronize-issues-to-jira[bot]]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-2561.

This message was autogenerated

[cjdcordeiro]

Hi @carlcsaposs-canonical

This is a valid request and something we've considered before. It isn't as straightforward to accommodate as we might think though, because we want this field to be deterministic, useful and relevant, so the following questions and remarks need to be considered:

• this field doesn't make sense without org.opencontainers.image.source
• what if the rock isn't being built from a git project? Sure we can make them optional, but that will make the resulting metadata less predictable
• what if the commit doesn't exist (e.g. was not pushed or was overwritten)? the rock would live out there with a broken reference that can not be fixed, since the rock is immutable
• what cases do we have for users interacting with this field? Can you elaborate a bit more on your case, explaining how you're using it? Is it just for inspection purposed, or are you doing any sort of label/annotation-driven policy placement in your deployments?
• what message does this field convey? Would consumers expect to fully reproduce that rock by following the source and revision annotations?
• finally, and most importantly: this field might generate confusion with the Store-generated "revision"

I'll follow up on this, but can't promise any immediate solutions.

[carlcsaposs-canonical]

Our use case for this is to be able to identify what source code (e.g. https://github.com/canonical/charmed-postgresql-rock) was used to build a rock (e.g. https://github.com/canonical/charmed-postgresql-rock/pkgs/container/charmed-postgresql/235196560)

So, if a customer has an issue, we can identify the source code for troubleshooting. This is analogous to what we do for snaps, where we tag the GitHub repository with the snap revision so that we can identify the source code for the version of a snap (e.g. https://github.com/canonical/charmed-postgresql-snap/releases/tag/rev115)

Currently, our only customers for our rocks are via our charms—and our charms pin exact hashes of the rock (example). So the process would look like: bug report from user with charm revision -> we are able to identify charm code via tag -> charm code contains rock hash -> identify rock source code

In the future, we may have customers that directly use our rocks, and the process would look like: bug report with rock hash -> identify rock source code

what message does this field convey?

the source code the rock was built with

from the spec: "org.opencontainers.image.revision Source control revision identifier for the packaged software." (https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys)

• this field doesn't make sense without org.opencontainers.image.source

we're happy to provide information to rockcraft to populate this in order to have support for org.opencontainers.image.revision

• what if the rock isn't being built from a git project? Sure we can make them optional, but that will make the resulting metadata less predictable

to support that, yes I think it would need to be optional

• Sure we can make them optional, but that will make the resulting metadata less predictable

• what if the commit doesn't exist (e.g. was not pushed or was overwritten)? the rock would live out there with a broken reference that can not be fixed, since the rock is immutable

Right now, (as far as I know) there is no way to identify the source code that a rock was built with (unless you maintain logs of releases or reinvent the wheel and create a custom metadata system outside of rockcraft and the OCI artifacts)

I believe this is a more pressing concern than commits not getting pushed or overwritten—IMO, it's better to have a system that works 95% of the time (and 100% of the time for repos following standard practices such as automated releases) than to not implement a system because it doesn't support non-standard cases. Also, in those cases, having a commit hash and not being able to find the commit is, IMO, no worse than the current system where you can't identify the source code in 100% of cases. Our systems for charms and snaps will also not work if someone releases a charm/snap without pushing a commit, but these systems have also been incredibly important and necessary for us to reproduce & fix bugs

with a broken reference that can not be fixed, since the rock is immutable

IMO, this is a feature, not a bug—org.opencontainers.image.revision, if present, should identify the source code the rock was built with. I don't think it should be possible to modify this value so that it points to a different commit than the commit the rock was built with

---

Also, I believe since these (org.opencontainers.image.source and org.opencontainers.image.revision) are standardized, supporting them may improve compatibility with other tools (e.g. GitHub, tools for security scanning) that are made for docker/oci images

[cjdcordeiro]

Given that you already do something similar for snaps, you can adopt the temporary measure of tagging your rock's repo after the rocks revision/digest (which is what is done today via the OCI Factory).

---

what message does this field convey?

it is not as simple as that. The OCI spec covers the nature of the field but we still need to find the value that best fits our story. Simply imagine the common case of a remote-build where another Git project is involved. So it's not about what the OCI says, but about what we expect users to do with such information.

---

Surely we could make this work tomorrow, but we can't introduce defining changes without covering all corner cases and properly doing our due diligence, because once we introduce this annotation, we shouldn't change its significance ever again. Again let's not forget that a "revision" means something different for our craft artifacts, so we definitely don't want to introduce this OCI annotation if later on it is going to conflict with our revision values.

[carlcsaposs-canonical]

rocks revision/digest

do rocks have a revision? or is it the same as the digest?

[cjdcordeiro]

That's up to the build pipeline to create. In OCI Factory, rocks do have a revision (which is different from their digest, and follow a snap-like incremental numbering approach) - e.g. https://github.com/canonical/oci-factory/pkgs/container/oci-factory%2Fprometheus - the _## suffix is the revision

[cjdcordeiro]

Hi folks. Just a quick update on this:

• We've revisited the annotation and original specs and decided (as alluded to above) that it would be premature to create this annotation, given that the store is incoming and our concept our a "revision" may be different. Eventually, it may very well be that this annotation turns out to be the actual git commit hash, but until we are sure of that (which will only happen once we have the stores in place), we cannot take the risk of introducing something that later one may change in nature, or compete and create ambiguity with another custom-made revision annotation.

Feel free to keep the issue open for revisiting in the future, but for now, I'm afraid there won't be any immediate action on this.