search

k0sproject / k0s

k0s - The Zero Friction Kubernetes
https://docs.k0sproject.io
Other
3.52k stars 355 forks source link

Reconcile Calico's IPPool Object #4061

Open pschichtel opened 7 months ago

pschichtel commented 7 months ago

Is your feature request related to a problem? Please describe.

I've changed my calico CNI configuration from overlay: Always to overlay: CrossSubnet. I'm aware that k0s does not currently support changing CNI providers, but I figured that I can just change the k0s config, see how for that gets me and cleanup the rest manually, given that I have prior experience with self-managing calico.

The state as of now seems to be that k0s reconciles the calico deployment partially based on the config change. I noticed that the daemonset gets updated correctly (it carries CALICO_IPV4POOL_IPIP with the overlay value) and possibly other objects in the cluster, however it does not seem to update the IPPool configuration (which eventually dictates how the routes published by bird will look like).

Describe the solution you would like

It would be nice if k0s would reconcile calico's configuration objects like IPPool as well.

Describe alternatives you've considered

Alternatively it would be neat to have some kind of warning message or other signal, that the CNI configuration changed, but not all changes have been applied to the cluster.

Additional context

No response

pschichtel commented 7 months ago

I'm fully aware that I'm sailing in dangerous waters here, so I'm perfectly ok with this getting rejected for being out of scope or not worth the risk of breaking something else in the reconciliation process.

twz123 commented 7 months ago

K0s actually reconciles the pod CIDR in the Calico manifests, but only in a way that's picked up on first installation. The subsequent config changes are ignored by Calico if the IPPool exists.

Changing this would probably mean to create the IPPool manually, instead of delegating this to Calico.

OTOH, is it safe to change the pod CIDR in a running cluster? K0s at least refuses to do it for the service CIDR and cluster domain...

pschichtel commented 7 months ago

Yeah changing the networks has probably a bigger impact, I've never attempted that, but changing the overlay mode was picked up immediately by calico (I watched the routes change)