search

kube-vip / kube-vip-cloud-provider

A general purpose cloud provider for Kube-Vip
Apache License 2.0
115 stars 27 forks source link

Multi cluster loadbalancing #41

Open soer3n opened 2 years ago

soer3n commented 2 years ago

Hi,

i'm looking for an on premise solution for managing services of type load balancer in multiple kubernetes cluster. This is currently not supported and is maybe out of scope of this project. I recognized that this is possible by configuring kube-vip to use wireguard.

Would it be an acceptable feature if the kube-vip-cloud-provider could create services in an upstream cluster and mapping node ports of downstream cluster nodes to the upstream cluster service ?

Are there big disadvantages compared to the wireguard solution with kube-vip?

Or should it be a separate provider for that because the logic would be completely decoupled from the current kube-vip-cloud-provider implementation?

I put already some time into an attempt to implement it in general. You can take a look on it at my fork. I've tested it successfully with an ingress-nginx-controller but not with tls enabled until now. If it is acceptable for you i would like to open a pull request.

There are some limitations until now. Endpoints for Upstream Services are limited to one subset and sessionAffinity is hardcoded. I've added an example configuration to examples directory.

thebsdbox commented 2 years ago

Yeah, I'd love to chat about what your plans are!

soer3n commented 2 years ago

Thank you for your reply. In general i thought that this project could do what is actually done by cloud providers for load balancers (kubernetes -> kubernetes instead of kubernetes -> cloud provider). Credentials are a kubeconfig which only needs permissions for objects in one namespace which can be limited.

This would be helpful if you want to run a managed kubernetes platform in an on-permise environment and you don't have a simple builtin solution for external load balancing.

In the meantime the platform solution we prefer is using metallb for that. The advantage of doing it in the proposed way would be that there is no software needed for allocating ips in provisioned clusters.

The ip-range configuration per namespace by a configmap is also nice for restricting allocation per cluster or a group of clusters.

If you have any other questions please tell me your prefered way for communication. I was not sure whether you want to discuss it in this issue or outside of it.