Consider use of Structured List to enable image acceleration

[kornelski]

The header is currently limited to one entry. This means it doesn't have room for multiple priorities within a response. Multiple server-driven and end-to-end priorities are very beneficial for serving images.

For example, in progressive JPEG:

1. Usually the first 177 bytes are the header that contains image dimensions. Delivering dimensions early is helpful for letting browsers perform final page layout sooner. Since it's just 177 bytes, it's fine to send the first 177 bytes with a very high urgency.

2. The next 12%-15% of the file is a first progressive "layer". Downloading this much is enough to draw a preview of the image. Delivering this part with higher priority than the rest of the images makes all images appear earlier on screen.

3. The first half of a progressive JPEG file is sufficient to render it at full resolution with a good-enough quality. The second half of the image may be given a very low priority ("4" in the waterfall below), as it contributes very little to visual completion of the page.

Here's an example of this technique in action:

The top filmstrip is with images using multiple priorities per response. The bottom uses one priority per response. Notice that the top version with multiple priorities is visually complete much sooner, in almost half of the time (because it delivers first half of all image bodies, before second halves of all image bodies). More on this technique.

The priorities have to be server-driven. Only the server can parse files ahead of time to know exactly when to stop sending. Although the client could try to send PRIORITY_UPDATE after receiving enough of an image file, the roundtrip time and buffering make this technique much less effective (even server-driven implementation is already very sensitive to backpressure).

The priorities should be end-to-end, because good prioritization of images requires parsing of image files, and HTTP proxies shouldn't be burdened with such complex format-specific task. If optimal priorities can be stored as a header for the response, then they can be generated by external tools and work with any HTTP implementation.

To make this feature possible, the priority header would need to be a structured list that can express urgency and progressive for byte ranges. In practice it doesn't have to be start-end range, but it's enough to express "from this point (byte offset) onwards, use the new priority".

[kazuho]

Thank you for opening the issue and elaborating on the use-case. I agree that assigning different urgencies based on byte ranges is beneficial.

OTOH, it is my view that we do not need a Structured List. Current design provides room for extensions.

For example, we could define a parameter named "range" that takes urgency, start and end offsets as the arguments (e.g., Priority: urgency=3, range=start=0;end=1000;urgency=1).

Considering the short deadline, it is my preference to start with something minimal that covers what we do today using the prioritization scheme of H2, then to define extensions to cover new use-cases.

[kornelski]

Can you have multiple range arguments?

priority: range=0-;urgency=1, range=177-;urgency=2, range=5000-;urgency=3

[LPardue]

I think the answer is no, based on our analysis of the SH spec in #89, see "Parsing a dictionary" in https://httpwg.org/http-extensions/draft-ietf-httpbis-header-structure.html#rfc.section.4.2.2

However, I think some type of "range parameter value is an inner list" syntax might be valid but I'm no expert. A strawman based on your example e.g

priority: urgency=1, range=(bytes=0-;urgency=1 bytes=177-;urgency=3 bytes=5000-;urgency=3)

[kazuho]

@LPardue

However, I think some type of "range parameter value is an inner list" syntax might be valid but I'm no expert. A strawman based on your example e.g

priority: urgency=1, range=(bytes=0-;urgency=1 bytes=177-;urgency=3 bytes=5000-;urgency=3)

Structured Headers does now allow hashtables to be used within a dictionary value. Therefore, it needs to be something like the following, that uses a list containing repeated sequence of byte offset and urgency.

priority: urgency=1, range-urgency=(0 1 177 3 5000 3)

[kornelski]

An Item of a structured list can be an integer with parameters. You could say the integer is the urgency, and then parameters can be used to specify other properties, such as progressive, and ranges.

urgency=1 could be nice and short:

priority: 1

urgency=1, progressive=1 could be:

priority: 1; progressive

and then, this would generalize nicely to ranges:

priority: 1, 3;progressive;start=177, 10;progressive;start=5000 

This creates one unified syntax, as opposed to urgency that is specified once outside, and then inside range-urgency using two different syntaxes. And it avoids creating a non-extensible microsyntax for the inner range-urgency list. Image headers are not progressive, and this syntax can express that, too.

[LPardue]

The main problem with a syntax that omits explicit names is that it bakes assumptions into the design, that's the main tradeoff here. A scheme that only cares about urgency and progressive can have a more optimised syntax.

But thow do you extend the behaviour to enhance prioritisation beyond urgency and progressive? This relates somewhat to the open question on the "extensible priority scheme" and what endpoints are actually trying to negotiate. One problem is not what is sent, but the defaults that are applied when an expected field is not sent, which Is reliant on the belief of scheme in use at that moment in time.

[kornelski]

@LPardue, I don't understand your point. I'm proposing a change from key=value, key=value to urgency; key=value; key=value, so AFAIK extensibility is not harmed. If new keys are added, they can be supported in both cases.

[LPardue]

My understanding of your proposal is that in this configuration, it means all priority headers must contain an urgency, that the field is not named and is not optional. That all parameters are properties of the urgency, not properties of the broader request. This restricts extending the behaviour of prioritisation to only augmenting the way urgency works.

In contrast, my understanding of the extensible priorities is that it introduces a generic framework for the expression of client intent via named parameters. The first instantiation of this framework is the urgency scheme (or whatever we want to call it), which is explicitly negotiated and requires the presence of urgency and progressive parameters; the absence of these parameters has a well-defined meaning - for requests it means use the default value, for responses it means no override of the client's priority. Imagine a scenario where a client issues a request, with default urgency but non-default progressive. I think the practical differences between your proposal and the current design in this scenarios is

current design request header

priority: progressive=?1

proposed design request header

priority: 3, progressive=?1

Another scenario, a client issues a request with non-default urgency and default progressive but the origin overrides the progressive parameter.

current design response header

priority: progressive=?1

proposed design response header

priority: <remembered request priority>, progressive=?1

I think this is an important observation (if true, I'm tired and might have made an error). Requiring state or context neutralises one of the benefits of our current design in comparison to H2 priorities - that processing of client intent does not depend on outside context or state. This could maybe get even worse; imagine two schemes that are the opposites of each other: urgency and anti-urgency. Both express their level using an integer but use opposite scales e.g 1 is most urgent in one scheme and least urgent in the other. An endpoint that receives the priority header 1; progressive needs to disambiguate whether this applies to urgency or anti-urgency, one way to do that is to carry the scheme alongside the priority (for example, in another header) or to include the scheme name directly inside. Either way, it doesn't seem very nice.

Now that said, using structured dictionary is somewhat gambling that people can extend the negotiated priority scheme using parameters in way that is compatible but, importantly, not coupled to other parameters. That might not come to be true and we may need to define new schemes for different parameter types. But at least if we use explicitly-named parameters, we can inspect each request in isolation, rather than needing to couple it with connection-level information.

[kornelski]

Yes, it's assuming urgency is always specified. If a new property is defined that doesn't make sense with urgency, it could say that if it's specified, then ignore the urgency. The urgency may even serve as a nice fallback for clients that don't support the new property. Syntactically, priority: default; anti-urgency=1 also works.

I think it's better to focus on concrete needs of prioritization we can predict and implement (like image prioritization that is in production today), rather than throw these away in favor of unknowns that are so far away we can't even predict what they may be. If requirements ever change dramatically, defining another header is always an option.