CC: build and publish encrypted image(s) for testing

[wainersm]

This issue is a spin off CC: Pull image and unpack of encrypted image on guest

In order to test pulling of encrypted image on guest, it obviously should use encrypted images. Quoting @stevenhorsman on issue #8111 :

""" What container image do we want to use? In CCv0 we use, docker.io/katadocker/ccv0-ssh built by https://github.com/confidential-containers/confidential-containers/tree/main/demos/ssh-demo, but should we take ownership of the image and build it ourselves like we do with the unencrypted test image: https://github.com/kata-containers/kata-containers/tree/main/tests/integration/kubernetes/runtimeclass_workloads/confidential? One thing worth noting with this is that to avoid sha caching/matching, we want to ensure that the content of our encrypted image is slightly different to the unencrypted one. We also want to ensure it's a multi-arch manifest container image so that we can easily work in multiple platforms. """

Requirements:

• The image(s) should be multi-arch
• The image(s) should be built by a Github workflow
• To avoid headache, the image(s) should be different from the unencrypted one and not made from generic images (e.g. busybox) - see more information on @fitzthum 's https://github.com/kata-containers/kata-containers/issues/8111#issuecomment-2021592428

Any thoughts?

[portersrc]

This issue came up in our weekly CoCo CI meeting. One open question to get this rolling: Which repo and folder should we use?

I proposed the CoCo infra repo. It could have a top-level folder called container-images with a Makefile and some supporting folders (for dockerfiles, configs, keys, etc.). If anyone has a better suggestion or strong preference, we can do that.

One consideration is that if we put this in kata-containers, then the test images will live in kata-containers/packages, versus confidential-containers/packages if we use infra, for example.

[wainersm]

Hi @portersrc !

This issue came up in our weekly CoCo CI meeting. One open question to get this rolling: Which repo and folder should we use?

It isn't an easy question I think.

Encrypted images are used on Kata's as well as guest-components' tests. CoCo recommends the use CoCo Keyprovider (see here) to encrypt the images. So perhaps they should be rebuilt and versioned (!) when the requirements (skopeo version, for example) change from guest-components repo?

Then we have the signed (encrypted or not) images which can be using different methods of signature. They are also used by those kata and guest-components tests. So where to build them?

All in all, I think we need to understand better where we are using those images, how they are built and finally define where they can be rebuilt.

I proposed the CoCo infra repo. It could have a top-level folder called container-images with a Makefile and some supporting folders (for dockerfiles, configs, keys, etc.). If anyone has a better suggestion or strong preference, we can do that.

One consideration is that if we put this in kata-containers, then the test images will live in kata-containers/packages, versus confidential-containers/packages if we use infra, for example.

Pushing to ghcr.io instead of quay.io seems the easiest path.

[portersrc]

All in all, I think we need to understand better where we are using those images, how they are built and finally define where they can be rebuilt.

Good point. Regardless of what we do, this should be answered. I'll scour coco and kata and report below.

[portersrc]

Where the images are pulled from and where we are using them

We have container test images on ghcr (coco and kata) and quay (coco and kata). We may pull from our own docker.io resources in some rare or stale cases. We test with third-party images in various places. Details below.

confidential-containers' ghcr has the following images (with various tags): busybox test-container-s390x test-container test-container-image-rs

kata-containers' ghcr seems to just host a basic test image here. This one is actually a good example of the workflow most people probably want. It codifies how the test container image is built, and it rebuilds and pushes regularly with github actions.

confidential-containers' quay is minimal now, I think, hosting only: test-images

kata-container's quay has images that I thought coco's ghcr was meant to supplant. Those are here.

But they seem to still be used in some places in kata and coco, e.g. here or here kata-container's quay also has this, but I don't see it used anywhere: encrypted-image-tests

We may use some docker.io account and resources still, e.g. see: katadocker/ccv0-ssh (maybe referenced here)

We test with third-party, public images in various places, e.g. see here or here.

How the images are built

With the exception of the the kata ghcr example mentioned above, I don't think the coco ghcr images are automatically built anywhere. (I didn't dig deeper to figure out their provenance, but i tried searching for "ghcr.io/confidential-containers/test" or searching for yaml files with just "ghcr.io/confidential-containers/" in their name; and I don't see any automation for them; there's also Ding's comment here about generating some of these himself.)

Where they can be rebuilt

I'm still partial to using the confidential-containers/infra repo. And then we add a CI job to trigger rebuilds (periodically, or when we touch one of those files).

Regarding the spread of container images used in testing: (1) The public, third-party container images that I noticed in some of the test code all look fine (i.e. it's desirable to test with these in some case); (2) but there are still some inconsistencies that we could try to handle after issue is closed, e.g. deleting the images from coco on quay after all references (including those linked above) are handled.