Improve Load Balancers

[mishushakov]

• [ ] Add rate-limiting
• [ ] Add custom rules (firewalls)
• [ ] Add option to automatically check incoming IPs in SPAM Databases (eg. spamhaus.org) and prevent them from accessing resources
• [ ] Make possible to make requests to Load Balancer IP (or domain) inside of the cluster
• [ ] Document, that Health Check endpoint should support PROXY protocol in order to work with LB, when the PROXY protocol support is activated
• [ ] Sync LB settings from dashboard to cluster and the other way around. Or just disable any web-based editing of the LoadBalancer, if it's managed from Kubernetes
• [ ] Provide more documentation and technical details

[sandhose]

Two things I'd like to add:

• the ability to re-use an existing loadbalancer. I recently accidently deleted my cluster, but the loadbalancers were still there ; I'd have loved to be able to attach that loadbalancer to a new cluster to avoid having to change all my DNS records and keep my IP
• correctly support externalTrafficPolicy: Local. Right now if you deploy a service with two replicas on a 3 node cluster, with a service type LoadBalancer with externalTrafficPolicy: Local, the 3 nodes are added to the LB, even though the pod only exist on two of them. Because of that, the LB is marked as unhealthy

[timoreimann]

Appreciate the input, thank you so much. 👏

@mishushakov a number of your points are specific to our LB product. I'll make sure to forward those to the responsible team so that they can look into triaging / prioritizing as needed. Will keep the ticket open to track the DOKS-specific parts.

@sandhose:

re: "the ability to re-use an existing loadbalancer": this is now possible on clusters running CCM v0.1.17 or later (which corresponds to DOKS versions 1.15.2-do.0, 1.14.5-do.0, 1.13.9-do.0, and later). What you basically have to do is specify the LB UUID on the corresponding Service object. See this section in the documentation for some more details.

re: LBs marked as unhealthy with externalTrafficPolicy: Local: unfortunately, this is a consequence of how Kubernetes implements the specific setting. We're aware that this is sub-optimal for users, and we have plans to provide a better UI/UX.

[timoreimann]

Also, please make sure to subscribe to #8 in order to stay up-to-date on developments around routing to external LBs from inside the cluster.

[mishushakov]

Hey, one more feature to the list

• [ ] Add alerting, monitoring and notifications (on downtime)

Recently, my node went down, because for some reason Kubernetes pulled a wrong version of traefik, which because of configuration incompatibility caused a 22hr downtime (since the health checks could not be obtained by a load-balancer)

There is currently no way for me to to know, whether a node went down, unless checking manually. The priority of such a feature is critical for me and i'd like to have it ASAP :)

Thank you very much for outstanding service & support!

[timoreimann]

Hey @mishushakov, thanks once more for providing valuable feedback.

Having good observability around load balancers sounds like a reasonable request. FWIW, you could build this yourself today by polling the configured health checks periodically and reporting into a system like Prometheus. I do understand how a built-in, zero overhead mechanism would be much more convenient to have as a user, however, so I'll make sure your request gets forwarded to the right people.

[mishushakov]

One more:

• [ ] Proxy Protocol for individual ports

[kyranb]

One more:

On demand TLS or support for letsencrypt certificates when not using DO's DNS servers.

[mishushakov]

Please see how render does automatic (wildcard) Let's Encrypt Certs