Adopting Envoy as a Service-to-Service Proxy at Reddit

Reddit previously used Airbnb's smartStack as their service mesh https://medium.com/airbnb-engineering/smartstack-service-discovery-in-the-cloud-4b8a080de619 But

Nerve and Synapse can only accept static configuration meaning and service registration updates required Puppet configuration changes and updates across their service fleet
Synapse's configuration writer for HAProxy provides only basic routing definitions
They had minimal observability for traffic going through HAProxy as it doesn't understand Thrift, which is Reddit's primary internal protocol

The first step that they took to deploying Envoy was to have it replace HAProxy for basic TCP proxying support. With Envoy and the new Thrift filter, they are finding greater observability at the network layer including request and response metrics that weren't available before without application code changes.

With the success of adopting Envoy at the proxy level to manage Layer 4 traffic, the next step in Reddit's plan is to the deploy Envoy's discovery service API backed by a centralized configuration store. Further reaching plans include investigating running Envoy at the edge as a replacement for using HAProxy as a load balancer for their core Reddit backend application and AWS ALBs for some of their external ingress points.

AdrianHu99 / NOTES

Adopting Envoy as a Service-to-Service Proxy at Reddit #22