search

kubernetes-sigs / cluster-api-provider-kubevirt

Cluster API Provider for KubeVirt
Apache License 2.0
108 stars 61 forks source link

ClusterClass incompatibility caused by KubevirtMachineTemplateSpec immutability. #264

Closed aamoyel closed 4 months ago

aamoyel commented 9 months ago

What steps did you take and what happened: Actually, at our company, we are using the CAPI ClusterClass feature to manage our clusters lifecycle. We saw that the capi-controller-manager tried to update KubevirtMachineTemplate when we update a field (like the memory of cpu cores) but we're facing this error :

"Reconciler error" err="error reconciling the Cluster topology: failed to reconcile MachineDeployment/worker-djv5f: failed to create patch helper for KubevirtMachineTemplate/worker-infra-fdjc7: server side apply dry-run failed for modified object: admission webhook \"validation.kubevirtmachinetemplate.infrastructure.cluster.x-k8s.io\" denied the request: KubevirtMachineTemplateSpec is immutable

Did you know why KubevirtMachineTemplate resources are immutable, is this for security, lifecycle of any other purpose ?

What did you expect to happen: The KubevirtMachineTemplate should be updated and a new machine set will be triggered by the cluster-api to rollout worker nodes.

/kind bug [One or more /area label. See https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt/labels?q=area for the list of labels]

aamoyel commented 9 months ago

I close this issue, the recommended way is at this page and all templates should be immutable : https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/change-clusterclass#changing-clusterclass-templates

aamoyel commented 4 months ago

Rencently we encountred the same error when ClusterClass was changed on a cluster. After some research, i've found this in the documentation: [Server Side Apply](https://kubernetes.io/docs/reference/using-api/server-side-apply/) implementation in ClusterClass and managed topologies requires to dry-run changes on templates. If infrastructure or bootstrap providers have implemented immutability checks in their InfrastructureMachineTemplate or BootstrapConfigTemplate webhooks, it is required to implement the following changes in order to prevent dry-run to return errors. The implementation requires sigs.k8s.io/controller-runtime in version >= v0.12.3.

(https://cluster-api.sigs.k8s.io/developer/providers/migrations/v1.1-to-v1.2?search=#required-api-changes-for-providers)

So, we need to implement the dry-run check in the "immutability webhook" first.