GKE Operator - Autopilot support

[nwmac]

Internal Reference: SURE-6518

Currently, we don't support GKE autopilot mode clusters. We have reports from users with significant usage that are specifically asking us to support autopilot for GKE. If we don't support GKE autopilot, we risk to push them towards different solutions for cluster lifecycle management.

More info: As detailed in https://github.com/rancher/rancher/issues/39052 and https://github.com/rancher/rancher/issues/41654 we do not currently support the import (nor provisioning) of GKE Autopilot clusters.

Attempts to import a GKE Autopilot cluster fail with an error of the format:

Violations details: {"[denied by autogke-node-affinity-selector-limitation]":["If not using workload separation, node affinity is not allowed on labels with keys: '[beta.kubernetes.io/os|http://beta.kubernetes.io/os]'; Autopilot only allows labels with keys: '[cloud.google.com/compute-class,cloud.google.com/gke-spot,cloud.google.com/gke-placement-group,topology.kubernetes.io/region,topology.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region,failure-domain.beta.kubernetes.io/zone,cloud.google.com/gke-os-distribution,kubernetes.io/os,kubernetes.io/arch,cloud.google.com/private-node,sandbox.gke.io/runtime,cloud.google.com/gke-accelerator,cloud.google.com/gke-accelerator-count,iam.gke.io/gke-metadata-server-enabled|http://cloud.google.com/compute-class,cloud.google.com/gke-spot,cloud.google.com/gke-placement-group,topology.kubernetes.io/region,topology.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region,failure-domain.beta.kubernetes.io/zone,cloud.google.com/gke-os-distribution,kubernetes.io/os,kubernetes.io/arch,cloud.google.com/private-node,sandbox.gke.io/runtime,cloud.google.com/gke-accelerator,cloud.google.com/gke-accelerator-count,iam.gke.io/gke-metadata-server-enabled]'."]}

[gaktive]

Related backend work that's complete:

• https://github.com/rancher/gke-operator/issues/136
• https://github.com/rancher/rancher/issues/41654
• https://github.com/rancher/gke-operator/pull/251

[gaktive]

Docs from GKE: https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview

[gaktive]

From an initial glance, this appears to be a toggle between managed nodes. Whoever starts here should sync with Highlander.

[gaktive]

The other factor to consider is that we'll be migrating GKE from Ember to Vue so this could happen at the same time.

[gaktive]

8967 is the GKE Ember migration ticket.

[mantis-toboggan-md]

Going off the comment in the updated rancher/rancher issue here it sounds like this is not ready for UI work yet. Is that correct @mjura?

[gaktive]

Still have to fully confirm this but it looks like UI & Highlander are blocked based on a Hostbusters ask in https://github.com/rancher/rancher/issues/39052