HTTP2 Max Streams aware client pooling

[howardjohn]

Is your feature request related to a problem? Please describe. We are dealing with an issue where pooled http2 connections are hanging due max streams limits. This is problematic for us, as all our streams are very long lived, so they end up ~indefinitely blocked.

We were hoping to get around this by tracking stream counts as part of the pool logic.

However, neither current or max stream counts are exposed.

Would it be feasible to expose those (at least Max - current can be tracked in our own code)?

Or is there an alternative solution we should look into?

Previous discussion brought up

No, the client won't start opening new connections when max streams is reached. This is because of RFC 9113 Section 9.1:

Clients SHOULD NOT open more than one HTTP/2 connection to a given host and port pair, where the host is derived from a URI, a selected alternative service [ALT-SVC], or a configured proxy.

Describe the solution you'd like A few options are plausible.

1. Pool can be configured to automatically open up new connections when connections are "full"
2. Hyper exposes max and current stream counts, so a pooling implementation can easily build this logic themselves. This could look something like fn is_open() { self.current() < self.max() }
3. Hyper exposes the current max streams (or, possibly, arbitrary http2 connection settings information. A pooling implementation can track the current streams themselves. I think this could be done reliably external to hyper, but there may be edge cases I am not considering.

Additional context Here is our current pool: https://github.com/istio/ztunnel/blob/a0d539ee6b395a823c31db8f382048dc1947bcb7/src/proxy/pool.rs#L68

Basically we are operating on http2::SendRequests

Some other libraries built on h2 expose similar: https://github.com/cloudflare/pingora/blob/20fd391f3e78c9349149f11fae94da9e4657478e/pingora-core/src/connectors/http/v2.rs#L239

[edwardwc]

Hey there @howardjohn,

If I understand correctly, you'd like to see how many streams are currently open?

[howardjohn]

Ideally the current and the configured max streams on the connection - the max is dynamic based on the client/server SETTINGS.

Without this we end up blocking when we have two many long lived requests to the same host

[edwardwc]

@howardjohn have you considered using an atomic counter that a structure increments or decrements when it is created and dropped to track the count?

[howardjohn]

@howardjohn have you considered using an atomic counter that a structure increments or decrements when it is created and dropped to track the count?

Yes, that is our current workaround. However, that only enables us to know how many active streams we have produced as a client. We cannot find the negotiated max streams on the connection because its not exposed by hyper. So today we are guessing no one will set it <100... but there is no guarantee, and it makes us extremely conservative

[dswij]

That's a fair ask. I'll get into it as soon as I have time to spare.