gRPC apps in Kubernetes scale correctly using gRPC client-side load balancing

[JamesNK]

Some load balancers don't work effectively with gRPC. L4 (transport) load balancers operate at a connection level, by distributing TCP connections across endpoints. Because gRPC calls happen on a single connection, all load gets sent to one server.

An example of this issue is Kubernetes. If a pod has multiple replicas, the gRPC connection will be made to one pod replica, and all calls will go to that pod replica. gRPC with default Kubernetes load balancing is broken.

There are two options to effectively load balance gRPC:

• Client-side load balancing
• L7 (application) proxy load balancing. For example, YARP and Envoy will distribute gRPC calls across replicas.

Proxy load balancing works today with gRPC in .NET. A lot of people will choose this option. However a proxy introduces complexity and performance penalties. Many customers have asked for support for client-side load balancing.

Other first-class gRPC implementations support client-side load balancing (grpc-go, grpc-java, C++). grpc-dotnet would aim to provide equivalent functionality.

Note: Changes to HttpClient maybe required to properly support client-side load balancing.

Reasons for client-side load balancing:

• A must have feature to support load balancing with Kubernetes replicas
• Provide a high-performance, proxyless alternative to using YARP/Envoy
• To maintain feature parity with other first-class gRPC implementations
• First-party customer demand

Dev features/work:

• [ ] Establish connection with HttpClient without making call. Also track status of connection - https://github.com/dotnet/runtime/issues/45246

[danmoseley]

@JamesNK as this is parented by a Theme, perhaps it should be an Epic (or be under a gRPC epic)

[danmoseley]

Same parent as https://github.com/dotnet/core/issues/5493 perhaps.

[danmoseley]

@JamesNK any requirements on the client networking stack here?

[JamesNK]

@JamesNK any requirements on the client networking stack here?

I'm pretty sure there will be. Post 5.0 release I'll talk to @karelz about it

[danmoseley]

@JamesNK it would be good to now gain clarity here on libraries asks.

[JamesNK]

@danmosemsft I've created an issue about the big System.Net.Http feature required here - https://github.com/dotnet/runtime/issues/45246

Will discuss with @karelz @scalablecory @geoffkizer.

[mairaw]

Bulk closing .NET 6 epics and user stories. If you think this issue was closed in error, please reopen the issue and update it accordingly.