Open geofflamrock opened 2 years ago
@geofflamrock I've just run into this on our Samples instance when I was configuring an example to show customers. How do you implement the workaround? Once the target was discovered and added, I encountered the could not find kubectl
error. Within Infrastructure, I changed the details so it used a different worker pool for the health check, however, when the deployment executed, it set the worker pool back to what it was. Is there something else that needs to be done?
@twerthi
Once the target was discovered and added, I encountered the could not find kubectl error. Within Infrastructure, I changed the details so it used a different worker pool for the health check, however, when the deployment executed, it set the worker pool back to what it was.
This is the way that the cloud target discovery works, which is a bit unfortunate for the workaround here, on the second deployment the target associated with the cluster gets rediscovered and updated, which reverts the worker pool back to the one used during discovery.
A couple of other things you might be be able to try here:
Octopus.Aws.WorkerPool
you can use to specify the worker pool that should be used during discovery, which should then also pass through to the target. At the moment we don't have an equivalent variable for Azure unfortunately.@geofflamrock
When updating the dynamically added target, I switched from running on the worker itself to using Execution Containers which allowed the health check to continue. All of the workers that are on Samples don't have have kubectl
explicitly installed, so we rely on the worker-tools
image. Is that an enhancement that might be able to be made? Supply an additional project variable Octopus.Worker.ExecutionContainer.Image
or something like that?
Team
Severity
No reports yet of this blocking customers
Version
2022.2.6249+, any 2022.3 version
Latest Version
No response
What happened?
When a Kubernetes cluster is discovered during a deployment using cloud target discovery, a health check is performed which requires
kubectl
to exist on the path. Some workers do not havekubectl
installed on them, including dynamic Ubuntu workers in Octopus Cloud and we recommend that an execution container is used on these workers. Kubernetes targets support setting a health check container image on them which supports this use of execution containers.The issue arises when discovering a Kubernetes target via cloud target discovery as we don't support setting a container on a discovered target, so the health check fails on these workers without
kubectl
.Reproduction
kubectl
and configure the use of an execution container. In Octopus Cloud Ubuntu dynamic workers do not havekubectl
.Target discovery will be performed and the cluster discovered, but the health check will fail as kubectl does not exist in the path and the execution container is not used.
Error and Stacktrace
More Information
Example of the error occurring:![image](https://user-images.githubusercontent.com/2915931/180103664-61d22923-9b66-4cb9-a907-e6615d9f449f.png)
Workaround
Workaround is to use a worker that has
kubectl
installed on it.