Load balancing in the new cluster configuration (combustable cronut) will require any load balanced services to have a health check endpoint. Returning a 200 at /healthz would be helpful to get this going.
How I've gotten to this point so far:
We've been attempting to get to a new cluster configuration that has private IP's and a NAT for internet access to allow customers to whitelist IP's. This requires a new cluster configuration on a new VPC because the routing is done on a subnet basis as well as the modification of in-place clusters or the adding of new clusters to private space is not possible per GKE config.
The new cluster has been created, but there are issues with private networking and ingress-nginx. See https://github.com/elastic/cloud-on-k8s/issues/1437 for why. StackOverflow results on this topic show it to be a brittle fix as well as a manual configuration we can't easily replicate.
We were looking to remove ingress-nginx from the data-plane-gateway topology anyway as it requires extra routing and latency as well as that there are no benefits we get from using it, such as routing, header modification, etc. The new cluster configuration has container level load balancing available, so this is definitely superfluous steps in routing to go through this nginx ingress and should be avoided. Side note, we'll need to convert services from nodeport to clusterIP to take advantage of this load balancing.
Google's external load balancers require health checks to be present and of the HTTP, HTTPS, or HTTP/2 protocols, TCP is not supported.
Load balancing in the new cluster configuration (combustable cronut) will require any load balanced services to have a health check endpoint. Returning a
200at/healthzwould be helpful to get this going.How I've gotten to this point so far:
ingress-nginx. See https://github.com/elastic/cloud-on-k8s/issues/1437 for why. StackOverflow results on this topic show it to be a brittle fix as well as a manual configuration we can't easily replicate.ingress-nginxfrom the data-plane-gateway topology anyway as it requires extra routing and latency as well as that there are no benefits we get from using it, such as routing, header modification, etc. The new cluster configuration has container level load balancing available, so this is definitely superfluous steps in routing to go through this nginx ingress and should be avoided. Side note, we'll need to convert services from nodeport to clusterIP to take advantage of this load balancing.