Look into a different pipeline structure

[bryce-anderson]

The current pipeline model is a pull-based pipeline model which brings some tradeoffs. Pros

• Clear semantics for backpressure
• Intuitive usage pattern

Cons

• Promises at each stage of the pipeline bring significant overhead
• The concurrency model is 'all up to the user'

A push-based pipeline model down to the tail, which can transition to a pull-based model gives the opportunity for both models at the cost of owning two models. How much does the second model cost? We should figure it out.

Thinning down the usage of the pull-based model (and more specifically, its usage of Futures) would facilitate a look into switching to the cats IO monad.

[rossabaker]

What is the strategy for back pressure? Consumers signal demand, a la reactive streams? Or, talking about something I know little of, like Netty's channel flags?

Would you want IO or be polymorphic in effect? The question there would be the overheard of the typeclasses. I suspect most http4s apps are using IO as a base monad, with Monix Task a distant second.

[bryce-anderson]

In my minds eye, the pipeline would most closely resemble what Netty does (I've grown to like their threading middle immensely). Within the pipeline I envision back-pressure being managed via explicit management of 'interests' and it will likely use callbacks/Promises for continuations. It will be a very stateful construct with a strictly prescribed concurrency model: the serial executor pattern like I've been using in the http2 stuff. Interaction with the pipeline from the application side (mostly for clients and protocols like HTTP/1.x which benefit from implicit back-pressure) will be managed by a sort of bridge: it will be essentially a HeadStage, and it's job is just to convert the two models, providing back-pressure implicitly via the read and write operations.

I haven't put too much though into what the monad interface of that HeadStage will be, but my go-to was going to be IO while reserving the right to go polymorphic should there be sufficient motivation. Thats a long way off: first I want to see if I can put together a minimal pipeline model.

[bryce-anderson]

Btw, I'd like to make this a 0.15.x project. I want to get 0.14.x into a position that it works for http4s proper again, and is in reasonable shape. I hope the new pipeline model won't be terrible invasive for http4s, but it's hard to say for sure right now, so I'm not trying to pound it out.