`finagle-http2`: Add a per session request limit option

[beatrichartz]

finagle-http2: Add a per session request limit option

Problem

When running finagle in a setup using HTTP/2 via application load balancers (ALB), the session is very likely subject to a limit on the number of requests. E.g. when running via NGINX, the request limit is 1K requests per connection by default. On AWS, ALB have a 10k request limit. This means that in setups that use finagle and ALBs for their L7 features, the session is subject to races happening on connection close regularly. Here's an illustration of the race happening after a limit of 10K requests has been reached:

Netty fires inactive channel for any request that is in flight for the closed session, leading Finagle to propagate a ChannelClosedException. There is no possible remediation for this race in Netty since Finagle manages sessions/connections.

This condition can be easily reproduced by running a Finagle server & client with an ALB like NGINX in between, sending concurrent requests up until the limit.

Solution

Allow configuration of a MaxRequestsPerSession option that aligns a session with underlying request limits. The number ofMaxConcurrentStreams is deducted to account for any requests in flight. When this number of requests is reached, mark the session as closed so it is shut down and a new session is created.

This option is off by default and therefore opt-in.

Result

When the configured number of requests is reached, the session is terminated and a new session is opened:

[CLAassistant]

All committers have signed the CLA.

[mattdickinson5]

hmm, for some reason this landed attributing the wrong person - apologies!

[beatrichartz]

No worries, glad it landed! 👍