Define OC/KUBECTL DIND images in Controller

[shreddedbacon]

Instead of defining the OC/KUBECTL DIND image overrides in the API / Webhooks2Tasks images, it also makes sense to define the overrides in the controllers themselves.

This would mean that all controllers would need to be updated with the image override though to prevent different controllers from deploying different DIND images

[shreddedbacon]

We currently have a few possible build images in lagoon. Since the controller can run in openshift or kubernetes, the default build image that would be supplied needs to default to one but be easily configurable for the other.

Proposing to default the build image to amazeeio/kubectl-build-deploy-dind:latest in the controller flags, that gets hardcoded into the controller as the default.

Helm values would define the actual arguments that get passed to the controller manager So I would see the helm value file being set to something like this, where we can define the actual builddeploy image with the correct version tag as required.

lagoonBuildDeploy:
  ...
  extraArgs:
  - "--metrics-addr=127.0.0.1:8080"
  - "--enable-leader-election=true"
  - "--override-builddeploy-image=amazeeio/<oc|kubectl>-build-deploy-dind:2.0.0" #or ${lagoonVersion}?
  - "--is-openshift=true" # would need to be set if the controller is running in openshift too, as there are a few differences

It would also get exposed as an environment variable OVERRIDE_BUILD_DEPLOY_DIND_IMAGE, so if that is set on the deployment/pod then it will be used instead of what is defined in the extraArgs

Any tests would need to be updated to make sure that when deploying the controllers, the extraArgs are update to ensure the correct image is applied to the controller when it's deployed. These are used in tests currently

buildImage = "172.17.0.1:5000/lagoon/oc-build-deploy-dind:latest"
buildImage = "172.17.0.1:5000/lagoon/kubectl-build-deploy-dind:latest"

Otherwise any other environment uses the amazeeiolagoon repo with the branch name as the tag.

buildImage = `amazeeiolagoon/oc-build-deploy-dind:${lagoonGitSafeBranch}`
buildImage = `amazeeiolagoon/kubectl-build-deploy-dind:${lagoonGitSafeBranch}`

[tobybellwood]

so - once we get the dockerhub naming sorted: uselagoon/kubectl-build-deploy-dind:latest will always point to the latest stable released image testlagoon/kubectl-build-deploy-dind:main will always point to the image made from the HEAD of the main branch

We should definitely override the version using the ExtraArgs for testing purposes.

[shreddedbacon]

Another one was the active/standby task image. I think this can probably stay being defined in Lagoon itself, rather than the controller. The only place these are called are by the API directly, so if we ever needed to overwrite the image, we would just need to patch or add the overwrite to the API directly rather than any controllers.

https://github.com/amazeeio/lagoon/blob/7bec36e5c410c2012d6700b38ca9e5b6b8c3bac5/node-packages/commons/src/tasks.ts#L1223-L1235

https://github.com/amazeeio/lagoon/blob/7bec36e5c410c2012d6700b38ca9e5b6b8c3bac5/node-packages/commons/src/tasks.ts#L1304-L1317

The idea here is that these are specific to tasks in Lagoon, and we can cater for openshift/kubernetes differences in the task itself. From the controllers perspective, there is no difference between a kubernetes or openshift task really except for the resources that get generated when being passed into the task image to be run.

Example, openshift active/standby generates the routemigration crd here https://github.com/amazeeio/lagoon/blob/7bec36e5c410c2012d6700b38ca9e5b6b8c3bac5/node-packages/commons/src/tasks.ts#L1209 and kubernetes uses ingressmigration here https://github.com/amazeeio/lagoon/blob/7bec36e5c410c2012d6700b38ca9e5b6b8c3bac5/node-packages/commons/src/tasks.ts#L1290

These crds then get passed to the active/standby task runner image to be processed in the destination cluster. The actual task image for kubernetes and openshift is the same, and the work they do in each cluster is the same, just the resource created is different.

In the future if we go further with performing custom tasks using this system, then it would get unwieldy adding every possible task image to the controller and having overrides. It makes more sense to keep this override within Lagoon, as only the API would need to know about the image, its much simpler to maintain.