CAPA: Kubernetes v27

[yulianedyalkova]

As Nikola already tried to create a GS cluster with Kubernetes release v27 and hit some issues, let's try to reproduce that and investigate how we fix the problems ahead of time.

Acceptance criteria:

• [ ] Ensure we can create Giant Swarm clusters with Kubernetes v27
• [ ] Do not release anything yet, we are mostly doing this for our own confidence

[weseven]

Reproduced the issue, the control plane isn't fully working due to the controller-manager crashing at the start.

This is cause by the removal of the aws in-tree cloud provider in kubernetes 1.27: https://github.com/kubernetes/kubernetes/pull/115838

We are correctly setting the cloud-provider as external in the cluster chart, and I think we deploy the aws cloud provider as a separate pod as expected, but we are also setting the external-cloud-volume-plugin which falls back to the in-tree cloud provider, which causes the issue.

Removing that line the controller-manager works as expected. I will test the changes with a test release and see if allocating volumes works correctly. We will also need to check whether any of the other providers actually need that flag.

[weseven]

Merged PR in cluster chart with the fix, removing the setting by default (providers can still define it through helm values) https://github.com/giantswarm/cluster/pull/242

Tested v25, v26, v27 and v28 cluster creation and v25 updated all the way through v28, everything works as expected