Support gRPC's Server-side Connection Management

[cherrot]

Is your feature request related to a problem? Please describe.

https://github.com/grpc/proposal/blob/master/A9-server-side-conn-mgt.md

I will not elaborate on the benefits of implementing server-side connection management (you can refer to the original proposal above), but I need to explain why we urgently need this feature: communication within K8s clusters via service endpoint is by default implemented as a L4 LB, which prevents gRPC clients from promptly detecting service Pod scaling, leading to unbalanced loads. (In my case I heavily use bidi stream RPCs so HTTP/2 connection is inevitable)

However, if server-side connection management is supported, it can achieve load balance by actively disconnecting and reconnecting to the underlying gRPC service Pods.

Here is a good article (and illustration) explaining this resolution on gRPC world:

How three lines of configuration solved our gRPC scaling issues in Kubernetes

Describe the solution you'd like

Implement Server-side Connection Management. Specifically, the MAX_CONNECTION_AGE and MAX_CONNECTION_AGE_GRACE parts of the original proposal.

Additional context

I saw a similar feature request in connect-es project: https://github.com/connectrpc/connect-es/issues/752

[jhump]

@cherrot, Connect implementations (not just this one, but others, too) generally rely on standard libraries for the transport layer. To that end, a Connect client or server does not know anything about the underlying connections/sockets. In the case of this repo, for Go, all that is handled by net/http. So I think you're looking for a custom net.Listener or perhaps HTTP middleware that could implement the above.

Having said that, you might be interested in a client-side solution: https://github.com/bufbuild/httplb. That package can be configured for clients to automatically recycle connections after they reach a particular age (instead of the server doing it) using the WithRoundTripperMaxLifetime option. Also appropriate when using an L4 balancer is the resolver.WithMinConnections function, which can cause a client to create multiple outbound connections, even if the target name resolves to only a single IP address.

Also consider using a headless service in Kubernetes. Combined with the "httplb" library mentioned above, the client would resolve all of the backend pods (since DNS queries for headless services provide all pod IPs) and then perform client-side load balancing across them.