Question: Can Envoy handle HAproxy's aloha protocol, and also do meta load-balancing?

[PenelopeFudd]

Title: Can Envoy do meta load-balancing?

Description:

We've got a number of CDN servers, and we want to set up Envoy as a load balancer that calls their healthcheck endpoints and perform all the usual load-balancing calculations, but instead of proxying the traffic, we want Envoy to return the ip address of the server it would have proxied the traffic to. The client would then bypass Envoy and connect to the server directly; which is why I called it "meta" load-balancing.

One idea was to have the healthcheck endpoint on each server return one of the strings (down, up, maint, ready, 50%, maxconn: 30) (stolen wholesale from haproxy), but if there's a better way to communicate how much load a server is willing to accept, great!

Relevant Links:

https://www.haproxy.com/documentation/aloha/latest/load-balancing/health-checks/agent-checks/ Varnish and Nginx only have a binary healthy/not-healthy approach, it seems.

Thanks for any insight you could lend!

[phlax]

cc @wbpcode

[wbpcode]

Change the weight by the health checker is not supported by the Envoy for now, although sound a good enhancement, but not sure if it's worth to introduce the new complexity. 🤔

---

perform all the usual load-balancing calculations, but instead of proxying the traffic, we want Envoy to return the ip address of the server it would have proxied the traffic to.

What is the request look like? A normal HTTP request or probe request?

[PenelopeFudd]

The client would send a normal HTTP request, and Envoy could respond with a temporary redirect to a backend server. The bulk of the traffic would then bypass Envoy. This works because the backend servers are publicly accessible.

I was looking at the Envoy documentation on circuit breaking, health checking, and degraded endpoints but couldn't see a way for the backend server to communicate how many connections it can accept on a second-by-second basis.

A partial workaround would be to have the backend servers detect if the request comes from Envoy, and if so, respond with a redirect to the backend server's real hostname or ip address. That solves the meta part of the problem, but it does send an unnecessary request (Envoy already knows the answer the backend will respond with).

Still need to figure out a way to dynamically change the weights of the backend servers in Envoy's configuration.

Does that help?

[PenelopeFudd]

So instead of connecting to the backend server, Envoy is just sending a redirect that has the same effect.

[wbpcode]

Still need to figure out a way to dynamically change the weights of the backend servers in Envoy's configuration.

I think there is no way for now (except the xDS)? 🤔

[wbpcode]

I will mark this as help wanted because it sound a reasonable enhancement to me.

[PenelopeFudd]

xDS looks to be all-encompassing, and can definitely handle this, if I can figure out the man pages. 😃

[Buffer0x7cd]

Is there any update on this ? I think updating the weights of backend by using server reported loads ( either as part of health check or response headers) would be really great since it can react to changes much faster without the need of running a central service. Looking a bit around seems like this design is already been used at few placess like ZuuL ( https://netflixtechblog.com/netflix-edge-load-balancing-695308b5548c) and bandaid ( https://dropbox.tech/infrastructure/enhancing-bandaid-load-balancing-at-dropbox-by-leveraging-real-time-backend-server-load-information) .