Multi-architecture Kabanero Operator

[kaczyns]

This issue will be used to discuss the design for installing the Kabanero Operator on platforms other than Intel/AMD. Several areas need to be considered:

• Producing binaries compatible with the different architectures
• Producing container images that contain one or more of those binaries
• Producing Kubernetes artifacts (ClusterServiceVersion, CatalogSource, Deployment, Pod) that reference the correct container image for the target architecture

There are some limitations or restrictions which will guide the high level of this design:

• There is a desire from somewhere to avoid using multi-architecture containers, since not all container registries support multi-architecture images.
• There is no support for selecting from multiple container images in any of the Kubernetes artifacts
• All container images are built using Travis in response to a GitHub release being created. This build must now produce containers for the different platforms. The containers should all be produced in the same build to guarantee that the content is the same between the platforms.

I believe these to be the decision points that will need to be addressed: 1) The kabanero-operator-registry contains the ClusterServiceVersion defining the Kabanero Operator. We will need to build one registry for each platform, and the Kabanero installed will need to select the correct one based on the target platform. 2) The ClusterServiceVersion contains the container image name for the kabanero-operator container. Since the ClusterServiceVersion is contained within the registry, and we're already going to select the correct registry, there is no additional choice to be made here, other than to make sure that the ClusterServiceVersion YAML has been modified to contain the correct image tag. 3) The kabanero-operator creates pods/deployments for the Landing Page, CLI Services, Admissions Webhook, Stack Controller and Eventing. The names of these images are currently coded into config/versions.yaml. Kabanero will need to come up with a naming scheme for container images such that the names in versions.yaml can be modified to load the correct image tag for the target platform.

[kaczyns]

Travis has multi-arch support - this is new since the last time we investigated multi arch support. https://docs.travis-ci.com/user/multi-cpu-architectures/

I added power to .travis.yml and was able to download the correct version of operator-sdk for power, and built the go code successfully.

Things die when we build the container images. I was able to verify that docker is pulling the correct UBI base image for power. However the Dockerfile for building the operator registry requires pulling an image that doesn't have a power (or zLinux) based container image. I know that Jeff had to build this on the side for zLinux. We will need to assess how we want to handle things like this, when the container images are not present. We could build the operator-registry project ourselves during the travis build and just reference it (essentially creating a new dockerfile that doesn't reference the parent), or could point at the images we make off on the side, or request that Red Hat build operator-registry for the other architectures.

[kaczyns]

It looks like v0.16.0 of operator-sdk will have s390x binaries attached to the release. It's not released yet but we should jump on it as soon as we can, and that should solve that piece of the puzzle for power and s390x. See operator-framework/operator-sdk#2526

[plainchanter]

First step: added ARCH= support to the Makefile so that builders can specify ARCH=s390x make {build, pull}-image ARCH=ppc64le make {build, pull}-image Default architecture is amd64.

I would propose that until we have a multi-architecture container solution, we adopt the convention that container for other architectures append a dash and the architecture to the image tag:

DOCKER_ID/kabanero-operator:latest-s390x DOCKER_ID/kabanero-operator:0.7.3-ppc64le

etc. This would allow use to add a consistent suffix wherever needed.