Scope of the Image Certification program: include conversion to OCI bundle?

[alban]

I see different kinds of software and services who could be interested by a certification related to the image-spec:

1. any containerized software, to be able to say "this image of e.g. Redis is an OCI Certified Image".
2. container runtimes, to be able to say "the containerd runtime can generate a OCI bundle from an OCI image as per the OCI Image Spec"
3. registry services, to be able to say "all images provided by quay.io/Docker Hub respect the OCI image spec"
4. builders, like "docker build", or buildah

If I understand the description in https://github.com/opencontainers/certification/pull/26, the focus of the OCI Certified Image program is on my point 1.

But parts of the image-spec are not only about the image format but about specifying the actions that a container runtime could do (my point 2). See image-spec/conversion.md:

• Extraction of the root filesystem from the set of filesystem layers.
• Conversion of the image configuration blob to an OCI Runtime configuration blob.

Shouldn't we have a certification program for this as well? The rules about the filesystem layers with the whiteout files can be tricky, so it seems useful to me to have a series of tests that container runtimes could run.

/cc @opencontainers/image-tools-maintainers @opencontainers/image-spec-maintainers

[wking]

But parts of the image-spec are not only about the image format but about specifying the actions that a container runtime could do (my point 2). Shouldn't we have a certification program for this as well?

In order to compliance-test this, you'd need to specify an API for feeding in an image and invoking the “default generated runtime configuration”.

Passing in an image is complicated by distribution being out-of-scope for image-spec (as you point out here). But an image-spec compliance tester could work around that by defining its own image-input API(s) (as runtime-tools does for runtime invocation).

The API for invoking the “default generated runtime configuration” would need to be specified in such a way as to not allow other conversion changes (which image-spec explicitly allows for the general conversion case), but that doesn't seem insurmountable.

[cyphar]

I'm not going to get into the full breadth of discussion we had about conversion.md (trust me when I say it took a while to get it merged). The reason for the whole "default generated runtime configuration" is that things like Docker don't provide access to the raw configuration generated, and do a bunch of modifications to it that would make conversion.md useless (unless we allow for those types of changes in a way that doesn't invalidate the whole point of standardising on what a "basic" conversion looks like). On the other hand, tools like umoci do conversions that match conversion.md.

Personally I think that this sort of testing would be useful (we do a lot of it within umoci already) but for certification you're immediately going to hit problems with "edge cases" where a tool that operates on images provides a bunch of opinionated defaults -- and doesn't provide the "default generated runtime configuration" (maybe because it doesn't make sense to expose that to the user). Maybe we should fail certification for that, but I feel it might make a lot of tools implement features purely for certification that users are never going to actually use.

[vbatts]

Personally I think this would be useful, despite however much it could be an interpretation of the implementation.

[jdolitsky]

Going to close as outdated. Please re-open if more discussion necessary.