search

k0sproject / k0s

k0s - The Zero Friction Kubernetes
https://docs.k0sproject.io
Other
3.15k stars 353 forks source link

k0s managed VIP for controlplane (kube api) #4031

Closed jnummelin closed 1 month ago

jnummelin commented 5 months ago

Is your feature request related to a problem? Please describe.

Using k0smotron and cluster API to provision clusters requires the usage of some sort of load balancers for control plane. In some common environments, e.g. VMWare, the infra does not really provide anything CAPI could use out of box. Thus user has to manage the LB part one way or the other. This creates lot of pain as seen e.g. in https://github.com/k0sproject/k0s/issues/1150

Describe the solution you would like

k0s could bundle in keepalived for virtual IP management. keepalived also support defining IPVS LB rules, so it could also act as a LB. Since keepalived is very battle tested and small, why not to include it into k0s itself.

So essentially k0s could spin up keepalived on the controller nodes. It can also sync up the IPVS backends from kubernetes service endpoints.

The VIP is configurable by user, k0s cannot realistically pick up by itself. And as keepalived does the address management using ARP, it has certain requirements. Such as being on either routable network or in the same subnet as controller nodes.

Describe alternatives you've considered

Let users manage all this. Not very convenient, especially when building the clusters using things like CAPI.

Additional context

No response

Skaronator commented 2 months ago

This would be wonderful to have in k0s. Talos (a dedicated kubernetes OS) provides this as well out of the box: https://www.talos.dev/v1.6/talos-guides/network/vip/

It might be useful to see how they do it since they are also open source (MPL)

https://github.com/search?q=repo%3Asiderolabs%2Ftalos%20vip&type=code

Skaronator commented 2 months ago

Looks like this is already in progress, exciting #4181