search

emissary-ingress / emissary

open source Kubernetes-native API gateway for microservices built on the Envoy Proxy
https://www.getambassador.io
Apache License 2.0
4.37k stars 686 forks source link

AWS-specific usage documentation for Ambassador #249

Closed plombardi89 closed 5 years ago

plombardi89 commented 6 years ago

One of the challenges with AWS is wrangling the large number of load balancer options and reconciling that with Kubernetes usage.

To recap... on AWS there are three major load balancing products:

Product Feature Set Status
ELBv1 (a.k.a "Classic ELB") L4 and limited (HTTP/1.1 only) L4, TLS offload Active
ELBv2 (a.k.a "ALB" or "Application LB" L7 only, TLS offload Active
NLB L4 only; No TLS offload Active

The history of these products is that "Classic" ELB came first and then ALB and NLB were added much more recently (2016 and 2017).

The ELB is the most generic and feature-rich of the three but it is also hinted at to be deprecated in the future by Amazon. As it is, the ELB system has not received any updates in a long time. Using an ELB as the fronting load balancer also means that:

  1. Healthchecks are global and unconfigurable per the current AWS Kubernetes annotation system.
  2. Stuff like AWS WAF cannot be used to prevent DDOS attacks

Original comment that spurred this issue for more context...

We can't just use ALB's though because Kubernetes does not support those as a type = LoadBalancer service.


ELB now known as "Classic LoadBalancer" ... support for limited-L7 HTTP 1.1, HTTPS 1.1, full-L4 TCP, TCP+TLS and can do TLS offload for you while managing all the certificate work. Doesn't support websocket tho unless you change the configuration to work in TCP or TCP+TLS mode. Even then you lose `X-Forwarded-For` headers, but can enable that with proxy protocol... but then you need to find a server that knows how to parse the TCP stream and handle the proxy protocol shit.

ALB is purely for L7 HTTP/HTTPS traffic but it supports HTTP 1.1 and HTTP 2 and WebSockets natively out of the box. However, because it's not a L4 load balancer the Kubernetes folks seem inclined to never make it work as `type = LoadBalancer` on a `v1.Service` so it will only ever work as an Ingress controller (https://github.com/kubernetes/kubernetes/issues/30518). AWS refers to ALB's internally as ELBv2 and the "classic" ELB as "ELBv1" they've also stopped adding features to ELBv1.

NLB is the third LB product from Amazon and is for the pure L4 TCP case. It doesn't suppport TLS off load though so it's kinda annoying.

So in effect what AWS has done is take the old ELB product and split it into two products... ALB and NLB.```

An early goal from us should be to at least document an optimal ELBv1 + Ambassador setup.
richarddli commented 5 years ago

We've added a bunch of AWS docs with Ambassador. Closing.