search

appcelerator-archive / ampnext-discussion

This is a placeholder repo for discussions and prototyping around the next version of amp.
0 stars 0 forks source link

use envoy for L7 traffic management #10

Open subfuzion opened 7 years ago

subfuzion commented 7 years ago

Proposal

Switch from haproxy to Lyft Envoy as the primary proxy.

Rationale

The core platform is primarily concerned with L7 request routing for HTTP and gRPC over HTTP/2 communication, followed by load balancing requests to swarm manager nodes. For our purposes, we need more intelligent service-to-service communication routing, the ability to enforce quotas and policies, and potentially the ability to automatically expose logical routes for service and container logs and stats, etc. Basically, we need governance at the microservice level.

Unlike either haproxy or nginx, Lyft Envoy serves as a proxy for both downstream and upstream communication, can run the same software at both edge as well as on each node, provides advanced load balancing features that are only supported by the commercial version of nginx, provides substantially more detailed statistics, and supports hot restarts. Envoy also defines a gRPC rate limiting interface configured by YAML files (which are automatically reloaded).

Envoy provides first class support for gRPC, correctly supporting HTTP/2 trailers to convey request status. Consequently, this allows us to exploit the benefits of a proxy instead of passing requests through directly to servers as we currently do. Envoy also provides a number of filters that bridge support for gRPC communication, including gRPC-Web, requests over HTTP/1.1 when the request content type is application/grpc, and transcoded requests from REST JSON API clients.

Recommended Reading

https://eng.lyft.com/envoy-7-months-later-41986c2fd443

ndegory commented 7 years ago

Unfortunately, it doesn't seem that the envoy Docker images scans the service metadata to update the configuration, as is done by the docker cloud haproxy image. That means we would have to implement that mechanism and build a custom envoy image.

subfuzion commented 7 years ago

Yes, we have trade-offs one way or another. I see the problem of gathering service metadata and updating the envoy configuration (which is I why I consider hot restarts a big plus) as a much simpler problem to solve than the related gRPC problems that Envoy solves compared to other proxy solutions.

However, this is not to say that Envoy is the right solution for us. We currently use dockercloud-haproxy; haproxy is familiar and well-understood, so we need to consider the impact that Envoy will have on our time as well as our use cases.

subfuzion commented 7 years ago

For updating, initial look at Envoy logical DNS seems like a good basis for service discovery and the REST API looks straightforward to implement...

ndegory commented 7 years ago

Notes from discussion in mid August: not a priority considering that