MAX_STREAMS and flow control

[ekinnear]

From https://github.com/ietf-wg-webtrans/draft-ietf-webtrans-http3/issues/22#issuecomment-1077698720:

figure out MAX_STREAMS, a way to limit the number of streams within a session

We've addressed the rest of the plan except for this part via the capsule DT, but there are still open questions about reordering of closed streams and their interaction with stream flow control limits via MAX_STREAMS.

[ekinnear]

If reliable resets happen in QUIC, then it should be possible to model this after how QUIC MAX_STREAMS works, and we can just look at the biggest numbered stream.

[DavidSchinazi]

Chair: discussed at IETF 116. Consensus in room was to punt this unless someone proposes a solution. More specifically, we'll keep this issue open until IETF 117. If someone proposes a PR (or separate draft) that adds flow control, we will give them agenda time at IETF 117 and discuss whether to include it based on the details of the proposal. If no one has proposed anything by then, we will close the issue with no action.

[DavidSchinazi]

Chair: discussed in editor's meeting. @ekinnear and @martinthomson wrote up a proposal. Action Item is with @vasilvv to review.

[martinthomson]

https://www.ietf.org/archive/id/draft-thomson-webtrans-session-limit-00.html is the proposal. We might also want to consider the inclusion of the WebTransport-Init header field to govern initial values for those. And then, see https://github.com/ietf-wg-webtrans/draft-ietf-webtrans-http2/pull/78 regarding settings to additionally govern these.

[DavidSchinazi]

Discussed at editor's meeting: plan is to discuss at 117. @ekinnear can you make slides please?

[DavidSchinazi]

Chair: discussed at 117. Lots of discussion in the room, but consensus in room on the following:

• we can't do pooling without flow control as that would cause starvation
• we want JavaScript to not get measurable changes when the session is pooled vs not pooled
• therefore, we can't disable flow control when pooling is disabled
• this is hard to implement
• implementers are focusing on other parts of WebTransport first
• for now, we'll wait on more implementer experience
• we will not publish the document without flow control

[ekinnear]

Discussed at length in editor's meeting. @ekinnear to file an issue with the W3C. Needs more discussion here, too.

[vasilvv]

I've been thinking about this a bit more, and I'm not sure draft-thomson-webtrans-session-limit actually solves the problem fully.

Consider the following situation. Imagine I have a connection where the client can open up to 100 streams (let's exclude actual HTTP requests for now), and it has 10 sessions. We want to avoid the situation where one session uses up all of the resources.

As far as I can tell, the nested flow control does not actually solve this problem. The server can tell the peer that it can accommodate 100 streams on every session, and that would be a perfectly valid thing to do (if, e.g., it's proxying to a backend, and backend's limit is also 100). The problem here is running into the connection-wide limit (that we already know).

Now, one possible scheme here would be to just do proportionate allocation. The browser sees it has 100 streams remaining, it can allocate 10 streams to each session.

Another option would be to separate stream limits for WebTransport sessions and stream limits for HTTP/3 stuff. I wrote a draft on one possible way to do this: https://vasilvv.github.io/draft-vvv-quic-namespaces/draft-vvv-quic-namespaces.html

[DavidSchinazi]

Chair: discussed in editor's meeting. Folks voiced concerns about various complex solutions to the flow control issue. There's interest in writing up a simple solution for now and allowing future extensions to be more complex if needed. @vasilvv has the action item to write up a minimal proposal that handles just enough flow control

[vasilvv]

So, the basic problem we are trying to solve here is as follows. Thanks to the QUIC flow control mechanisms, an endpoint has a limit on how many streams it can open. That limit is shared by multiple WebTransport sessions, as well as non-WebTransport streams (such as regular HTTP requests). The endpoint has to limit the number of streams in each session, and also limit number of concurrent HTTP requests so that they don't starve out WebTransport data streams.

We can model this limitation as

max_concurrent_streams = max_streams_per_session * max sessions + max_requests,

where we already know max_sessions from SETTINGS_WEBTRANSPORT_MAX_SESSIONS. We kind of know max_concurrent_streams from the flow control window, though that is not necessarily always the number we want, since the peer can use different strategies for updating window (e.g. start low and ramp it up exponentially).

My current proposal is to add a setting for WEBTRANSPORT_MAX_BIDI_STREAMS_HINT and MAX_TOTAL_BIDI_STREAMS_HINT, that tell us the values of max_streams_per_session and max_concurrent_streams; max_requests can be inferred from the formula above.

Worked example: assume that a server is willing to accept a max of 100 concurrent streams. It sends a MAX_STREAM_ID of 400 to indicate that; it also sends MAX_TOTAL_BIDI_STREAMS_HINT of 100 to indicate that this is what the client should expect going further. The server can set MAX_SESSIONS to 3, and WEBTRANSPORT_MAX_BIDI_STREAMS_HINT to 30, meaning that the three WebTransport sessions can consume a total of 90 streams, leaving a headroom of at least 10 streams to regular HTTP traffic.

Note that those are called "hints", because those are not directly enforced; if the client does not enforce those, some parts of the connection may starve the others, but this is not a concern since (1) all of those share the same origin, and (2) they still have to fit into the peer's overall limit, which is strictly enforced by the protocol.

The above mechanism can be extended to other types of streams, as well as MAX_STREAM_DATA (it's actually easier for some of those, since e.g. the only server-initiated bidi streams in H3 are from WebTransport).

[enygren]

We should think about this from an adversarial perspective as well. Recent experience with HTTP/2 Rapid Reset (and similar) attacks demonstrate further how servers need to be able to bound their resource usage. We'll want to make sure servers can defend themselves against a range of creative attacks in a way that doesn't impair interoperability.

[DavidSchinazi]

Chair: discussed in the WG session at IETF 118. No consensus on a path forward at this stage. We heard:

• we don't need flow control
• we should do flow control hints
• we need a reliable flow control solution

All of which happen to be incompatible.

We'll need to keep discussing this. The chairs will consider potentially setting up an interim meeting focused on this topic.

[DavidSchinazi]

Assigning to @vasilvv since @ekinnear has written the PR so the next step here is for @vasilvv to review #166.