HTTP/2: check max-concurrent-streams setting

[the-mikedavis]

When under very high load with many concurrent requests, some requests may fail with

{:error, %Mint.HTTPError{module: Mint.HTTP2, reason: :too_many_concurrent_requests}}

From the Mint.HTTP2.request/5 docs:

In HTTP/2, opening a request means opening a new HTTP/2 stream (see the module documentation). This means that a request could fail because the maximum number of concurrent streams allowed by the server has been reached. In that case, the error reason :too_many_concurrent_requests is returned. If you want to avoid incurring in this error, you can retrieve the value of the maximum number of concurrent streams supported by the server through get_server_setting/2 (passing in the :max_concurrent_streams setting name).

It looks like this check is very cheap on the Mint side: it needs to check against a field in the conn. See https://github.com/elixir-mint/mint/blob/367230ff0565df69125f93204fcb496774bf69a9/lib/mint/http2.ex#L1065-L1070. There's also Mint.HTTP2.open_request_count/1 for checking the currently open number of requests for a conn.

I think it'd be ideal that if a Finch.HTTP2.Pool process has maxed out its allowed concurrent streams, the request gets redirected to another available Finch.HTTP2.Pool process if there is one.

I'd be willing to give this a look when I have the time :+1:

[the-mikedavis]

Found this stress-testing in this PR: https://github.com/sneako/finch/pull/158#issuecomment-986310868

[nallwhy]

@sneako I really need this feature. Is there any progress on it? Or can I check open_request_count by myself with Finch? Or can I limit the max concurrent request manually?

[nallwhy]

I checked the :max_concurrent_streams with the code below.

{:ok, conn} = Mint.HTTP2.connect(:https, "api.openai.com", 443)
Mint.HTTP2.get_server_setting(conn, :max_concurrent_streams)

[sneako]

I'm not aware of any ongoing work on this feature at the moment. PRs are welcome!