A new server archtecture

[kazu-yamamoto]

GHC 9.6 provides listThreads finally. Watching the number of threads in http2 using listThreads revealed that a lot of threads are instantly spawn due to forkAndEnqueueWhenReady. One waits for TBQ and the other waits for stream window.

To reduce the number of threads, I would like to propose another architecture for servers. Stop using the worker pool model. Rather, one thread is spawn for each stream. The number of threads follows maxConcurrentStreams.

Each designated thread feeds an output chunk to the output queue from a file or streaming data repeatedly. I guess that the complicatedDynaNext is not necessary anymore. This should drastically simplify Sender. That is, Sender just packs output chunks from the output queue.

Stream window should be taken care by each designated thread. Connection window should be taken care by Sender.

@edsko @FinleyMcIlwaine What do you think?

[kazu-yamamoto]

DynaNext cannot be removed because Sender owns confWriteBuffer. Works cannot manipulate the buffer. But I think it is still possible to reduce the number of threads.

[edsko]

I think in principle this makes sense. As you already point out, having threads per stream avoids the deadlock that we ran into in https://github.com/kazu-yamamoto/network-control/pull/4 . Indeed, in grapesy (my gRPC library that sits on top of http2 which has been motivating all the PRs that we've been submitting the last year), I spawn two threads for each stream, one to read incoming messages and one to send outgoing messages.

The devil is always in the detail of course; some rethinking of the architecture might be useful, but without actually thinking it through completely it's hard for me to really say whether this will be an overall improvement or not.

That being said, I think if the goal is simplification of the model, and making the code easier to understand, I'm all for it; if the goal is merely to reduce the number of threads, then I'm less convinced that it's terribly useful; threads in Haskell are cheap enough. Indeed, the number of concurrent streams is significantly higher than the number of workers currently employed by http2 (*).

(*) Caveat: this is actually only true for non-streaming bodies; grapesy exclusively uses streaming requests and responses, which means that we actually have as many workers as there are streams, because http2 will spawn a new worker each time a streaming response is initiated.

[kazu-yamamoto]

Done via #130.