HTTP2 Stream Priority Support in Jetty 9.4.38

[dev-chirag]

Hi,

Does Jetty 9.4.38 supports HTTP2 Stream weights and dependencies? I can send these in HTTP2 client, but I couldn't see how it's managed for the Jetty server.

[sbordet]

HTTP/2 stream priorities are not implemented, and will likely not be implemented. Very few servers implement it (too complicated and burns CPU instead of downloading the resources).

HTTP/3 has removed it from the specification, so it's pretty much a dead concept at this point.

[dev-chirag]

Thanks Simone. I saw your earlier explanation of not supporting it and that makes sense.

I am actually working on a use case where request prioritization is required. Just out of curiosity, do you think QOSFilter can be be loosely related to stream priority (in terms of request prioritization) and be used for HTTP2 as well?

[sbordet]

I am actually working on a use case where request prioritization is required.

I'd love to hear about it.

QoSFilter may be used to implement some sort of prioritization, but only when the filter reaches max capacity (i.e. only requests that exceed the max number of concurrent requests are suspended and prioritized).

You may try to implement something along those lines, but it's going to be a lot of suspend, shuffling around, etc. and by the time the prioritization algorithm is done, the server could have already served the resource, not to mention that the client may re-reprioritize in the middle of a single resource download.

Imagine a server having to do this for the 10K clients connected to it -- not a good use of server resources.

That's why I'm interested in knowing why in your case prioritization is required.

[dev-chirag]

I am implementing an endpoint for 5G HTTP2 interface that accepts all sorts of requests but must prioritize certain critical requests.

According to 3GPP, the critical requests are identified by a custom HTTP header that carries the priority value. In overload conditions, a 5G endpoint is expected to allocate maximum resources to the critical requests.

Though, HTTP2 stream prioritization is also required of a 5G endpoint, it might not be as critical as this above use case.

[sbordet]

Seems weird. In overloaded conditions, there may be a long time before the server processes the stream, parses its headers, and eventually calls the application to process it, to figure out it was critical.

The same goes if HTTP/2 prioritization was implemented.

Would it not have been better to send high priority requests to a different server port, with a high priority thread pool, only dedicated to critical requests?

[joakime]

This is the spec the OP is referring to - https://www.etsi.org/deliver/etsi_ts/129500_129599/129500/16.05.00_60/ts_129500v160500p.pdf

And the field 3gpp-Sbi-Message-Priority (defined in Section 6.8)

A quick read of that spec means it's not actually using the HTTP/2 priority mechanism, but instead wants a field in the payload of the data (JSON) to interpreted to know how to set the HTTP/2 prioritization. If that's true, it's broken by design, there's no way to have HTTP/2 protocol level prioritization based on HTTP/2 request payload data (it's far too late).

[dev-chirag]

My understanding of this spec is that both are needed in an endpoint. Though it doesn't clearly lists the use case for HTTP2 stream prioritisation, it's a mandatory requirement.

I am specifically interested in the first use case. One alternative that I have in mind is a congestion control procedure on top of Jetty endpoint where it can proactively limit the queued concurrent requests and starts rejecting low priority request with least processing.

[gregw]

If the priority header is left in the request, then it is moderately simple to modify QoSFilter to use that header when choosing which requests to prioritise. But as Simone says, until some concurrency limit is reached, all requests are treated equally and there is no throttling. So if your limit is 100 and 100 low priority requests have arrived then the 101st request will have to wait even if it is high priority. However, once one of the 100 requests complete, then the highest priority waiting request will be the first woken up and let to proceed.

It would also be possible to write a variation of QoSFilter that has multiple limits - one for low priority and another for high priority requests. So in the case above if 100 low priority requests had consumed all the low priority tokens, perhaps there would be another 20 high priority tokens defined that were only available to allow high priority requests to proceed. If you really wanted to, it would also be possible to boost the thread priority of any request dispatched with a high priority request, but might make things worse depending on the application e.g. if low priority requests are holding locks needed by a high priority ones.

It is harder to do all the above on H2 stream priority because that is complex definition of relative priorities rather than absolutes. I've seen proposals to use H2 stream priority that relies on creating several fake request that are long held, which are used as absolute priorities against which other streams can be ranked.... horribly complex. If the client is known and how it uses stream priorities is known and there is some agreed way to convert a relative priority to an absolute priority (eg any stream with a priority is considered high priority), then we could do similar actions above based on it... but if the header is there, then best to use that.

In summary, I think it is possible to at least "look like" the server is implementing some kind of priority mechanism. Whether it will actually work in practise is hard to say. Once a server is really overloaded it is difficult to make it start executing extra code in order to work out which task to run. In order to achieve results, you will need to define a low limit for "overloaded", so that even when overloaded with low priority requests, it has spare capacity to process high priority requests.

[joakime]

The HTTP/2 priority concepts are being removed from the spec today.

https://github.com/httpwg/http2-spec/pull/795

[sbordet]

@joakime wow, so we have been right from the beginning 😄

[rmarx]

It might be important to note that they are being replaced by a simpler scheme that will become recommended for both HTTP/2 and HTTP/3. This new scheme is also based (mainly) on HTTP headers, and thus similar to the 3gpp approach. This should be much easier to implement and cost fewer CPU resources than the heavier original HTTP/2 scheme.

It's not up to me to say what you all need to do of course, but I think it would be good if Jetty would support this new system over time. There are many use cases that require some level of prioritization to get optimal performance (Web page loading main among them, nicely explained by cloudflare here) and the basics (of the new scheme at least) are easy enough to implement efficiently.

[sbordet]

@rmarx from the point of view of a server that is handling 100k+ connections, each with 10+ streams, spending CPU cycles in prioritization seems sub-optimal.

It is perhaps true that for a single page priority matters; I would be interested in knowing whether 100k+ pages, cumulatively, will be delivered faster (by how much) and with less resource consumption with or without priority processing. I have not seen such studies yet; what I have seen all concentrate on the client point of view and for a single page, assuming the server has nothing else to do than serving the resources for that page. Have you seen some studies along the lines of "effects of prioritization when the server is busy sending resources to 100k+ browsers"?

In any case, we will tackle again prioritization if and when it's finalized.

[rmarx]

I think that's a question about where you want to optimize/be efficient more... many claim that optimizing Web performance can lead to a significant rise of revenue, as users are more engaged with faster loading pages. The fact that you can handle fewer connections per server instance might be worth it, if that additional revenue (more than) compensates for the cost of additional servers. That's of course a more difficult sell from the perspective of the server software provider and it's not proven that prioritization alone can have that type of impact on all pages etc.

It also somewhat depends on your default approach. If you send responses in a FIFO manner, then you'll do much better for Web page loading performance than if you'd round-robin between different active streams. I don't know what Jetty does today.

I don't have data on "effects of prioritization when the server is busy sending resources to 100k+ browsers", but for Web perf in general the network latency and processing order of resources (driven by prioritization) typically matter much more than Time to First Byte from the server for example.

In conclusion, while I understand your argument, I'm not sure it holds for the Web page loading use case (though proving that is a different matter ;))

[NageshSoni]

While implementing the QoSFilter I read in Java Doc of QoSFilter that this filter could be deployed with a maxRequest limit slightly smaller than the containers thread pool. How to decide on maxRequests?

[sbordet]

@NageshSoni your comment has nothing to do with this issue. Please open a new "Question" issue.