Merge prerender into preload and introduce "loadpolicy"

[igrigorik]

preload and prerender have high functional overlap and subtle difference in processing policies. Further, current separation makes it impossible to implement some load+processing policies.

Proposal: merge prerender into preload and expose necessary fetch and execution controls via new loadpolicy attribute. loadpolicy consists of a space-separate set of keywords:

• critical: indicates critical resource that must be fetched immediately and with high priority.
• optional: indicates an optional resource that may be used on current or next navigation and should be fetched with lower relative priority.
• inert: fetch, do not process.
• prepare: fetch and if possible preprocess/prepare the response - e.g. decode an image, prerender an HTML document.

Processing:

• If not specified, the default loadpolicy is critical inert.
• When combined, critical overrides optional, but combination can be used to indicate that the resource is critical for current page and is likely to be used on next navigation - i.e. UA should not cancel fetch across navigations.
• When combined, prepare overrides inert.

Examples:

<!-- defaults to "critical inert" -->
<link rel="preload" as="image" href="//example.com/some.js">

<!-- critical image that is used across pages, prepare if possible -->
<link rel="preload" as="image"
      href="//example.com/logo.jpg"
      loadpolicy="critical optional prepare">

<!-- prefetch without execution (inert by default) -->
<link rel="preload" as="html"
      href="//example.com/next-page.html"
      loadpolicy="optional">

<!-- prerender -->
<link rel="preload" as="html"
      href="//example.com/next-page.html"
      loadpolicy="optional prepare">

---

There are multiple benefits to merging these hints:

• Clean distinction between preconnect vs. preload: preconnect is used when only host is known, preload is used to define fetch and processing policy when resource URL is known.
• loadpolicy is more expressive and enables new processing strategies - e.g. resource is used across pages; resource is critical and should be 'prepared' if possible.

Thoughts?

[yoavweiss]

Merging preload and prerender makes a lot of sense.

Since we have probability to determine the likelihood that a certain resource will be needed, why do we need authors to define inert vs. prepare? Can't they just trust the browser to Do The Right Thing™?

The signals I see that we need to delegate to authors for merging preload and prerender are "we need this in current navigation", "we need this in the next navigation" and (probably) "we need this in current navigation, but also in the next one, so don't terminate the download if the user navigates". Are there other use-cases that need more signals?

critical and optional can handle the above use cases (if optional is defined as "next navigation"), but at least to me that definition is not intuitive.

[igrigorik]

Since we have probability to determine the likelihood that a certain resource will be needed, why do we need authors to define inert vs. prepare? Can't they just trust the browser to Do The Right Thing™?

Preload is a declarative fetch, as such its default job is to fetch the resource and place it in appropriate cache(s), no further processing is applied - i.e. its a much smarter version of todays "rel=prefetch". However, we do have existing examples where fetch is combined with further processing - e.g. prerender as its implemented today: fetch, process HTML, fetch subresources, etc. There is no reason why similar techniques can't be extended to other content-types - e.g. decode an image ahead of time, etc.

The distinction between prefetch vs prerender is mapped to inert vs prepare. We default to inert, but allow the developer to hint the browser that it should try to apply some preprocessing via "prepare" -- what that actually means depends on the content type; how far the UA goes with it is also deferred to it (e.g. it may skip the processing part due to resource constraints)...

You could argue that the whole thing could/should be left to UA, but I'm a little uneasy with premise of "preload is a declarative fetch, and it may sometimes prerender/process, but you don't really know when". I do think it's useful to provide this opt-in / opt-out behavior, but I'm open to being convinced otherwise.

Probability would apply to "optional", as an additional signal to indicate likelihood of using the requested resource in current or next navigation.

The signals I see that we need to delegate to authors for merging preload and prerender are "we need this in current navigation", "we need this in the next navigation" and (probably) "we need this in current navigation, but also in the next one, so don't terminate the download if the user navigates". Are there other use-cases that need more signals?

Not that I've heard of yet. Current, next, current+next should cover it.

P.S. If you have suggestions to replace "optional", lmk.

[bizzbyster]

I think it'd be better to have a clear indicator of needing for current page or needing for subsequent page. Critical/optional blurs this distinction because optional could also be used for a sub-resource on the current page.

[igrigorik]

@bizzbyster what's the difference between optional for current page vs next page? Under the hood, they would be initiated with the same (low) request priority, and so on. I don't see how the distinction is useful. Also, note that a binary signal can't account for critical+optional case: you need it on this page and you will likely need it on the next page.

[bizzbyster]

The UA might have reason to believe the user will not in fact navigate to the next page, perhaps based on an analysis of the user's browsing history. The web server may not be privvy to this information. There is a probability that a user will navigate to the next page. And there is another probability that a given sub resource is needed to render a given page. We lose information if we blend these.

[igrigorik]

Note that the site developer/server can still encode this information by computing a conditional probability - e.g. optional for current page vs. optional given user navigates to the next page vs. union of both for resources that may be used on both pages.

Strictly speaking, yes, the browser has slightly less control, but I think this is a fairly minor concession, in favor of a simpler and more flexible model. That said, I'm open to ideas.

[bizzbyster]

Okay. How about this? 1) probability supported in all cases. 2) Inert/prepare as per your above. And 3) in the place of critical/optional I'd suggest current/subsequent to indicate whether the resource is needed to render what the user has already requested (current) versus what the user may request in the future (subsequent).

[igrigorik]

"needed to render what the user has already requested (current) + probability" are at odds, unless Pr=1.0.

[bizzbyster]

The probability is the probability that it will be needed to render the current page if "current". The probability is the probability needed to render a subsequent page if subsequent. There is no attempt to express the probability that a user will visit a subsequent page.

[yoavweiss]

The distinction between prefetch vs prerender is mapped to inert vs prepare. We default to inert, but allow the developer to hint the browser that it should try to apply some preprocessing via "prepare" -- what that actually means depends on the content type; how far the UA goes with it is also deferred to it (e.g. it may skip the processing part due to resource constraints)...

You could argue that the whole thing could/should be left to UA, but I'm a little uneasy with premise of "preload is a declarative fetch, and it may sometimes prerender/process, but you don't really know when". I do think it's useful to provide this opt-in / opt-out behavior, but I'm open to being convinced otherwise.

I'm comfortable with "inert" and "prepare" being an opt-out and opt-in mechanisms, as long as the lack of either gives the browser freedom to do whatever it see fit based on its available resources.

P.S. If you have suggestions to replace "optional", lmk.

"lowpriority"? "non-critical"? Dunno, but let's bikeshed this some more

[igrigorik]

I'm comfortable with "inert" and "prepare" being an opt-out and opt-in mechanisms, as long as the lack of either gives the browser freedom to do whatever it see fit based on its available resources.

@yoavweiss mmm, interesting.. So, you're suggesting that we make prepare the default, to allow the browser more processing flexibility if resources are available, and if you want to explicitly opt-out from processing (e.g. don't attempt to prerender the page), then use "inert"? I'm OK with that.

---

Perhaps we can simplify this whole thing a bit further:

• Drop the default prepare keyword; developer opts-out from processing via inert.
• Drop the default critical keyword used to indicate current page resource.
  • Use low-priority to indicate resource is intended for next navigation.

For "current page + next page" policy case... drop it, since that's a statement that depends on you knowing what the next page is, of which there may be multiple! To address this, you'd need to restrict the "persist this fetch for this URL pattern", which gets rather complicated fast. Instead, I wonder if this is simply a heuristic that can be deferred to the UA - e.g. if preload is in progress and a navigation to the same origin page is invoked, then let it persist to see if it can/will be used on next page.

The net outcome is that we have an even simpler + cleaner API:

<!-- high-priority fetch + preprocess if possible -->
<link rel="preload" as="image" href="//example.com/logo.jpg">

<!-- high-priority fetch + processing opt-out -->
<link rel="preload" as="html" loadpolicy="inert"
      href="//example.com/document-fragment.html">

<!-- low-priority fetch + preprocess if possible (i.e. today's prerender) -->
<link rel="preload" as="html" loadpolicy="low-priority"
      href="//example.com/next-page.html">

<!-- low-priority fetch + processing opt-out (i.e. today's prefetch) -->
<link rel="preload" as="html" loadpolicy="low-priority inert"
      href="//example.com/next-page.html">

Probability applies to both high and low-priority fetches:

• for high-priority, the default is 1.0 (UA must fetch) but can be set lower to indicate a speculative fetch (UA may fetch) for current navigation context.
• for low-priority, the default is deferred to the UA (e.g. based on past nav/resource history), but can be set by the developer. All low-priority fetches are "UA may fetch", regardless of probability.

WDYT?

[bizzbyster]

I like it. My only tiny complaint is the term "low-priority" might get confusing in the context of other prioritization schemes. For instance, CSS is typically treated with a "high priority" and HTTP/2 allows for dependencies/weights, which is essentially another priority scheme.

I think "non-critical" is better. Or maybe there are other ideas? "background"?

[igrigorik]

Yes, that's true, "priority" is an often (ab)used and is easily misinterpreted. In terms of communicating the intent, perhaps "speculative" in place of "low-priority"? After all, anytime you're requesting a resource for the next page you are speculating that the user will head there. The name itself is on the long side, but I prefer clarity over brevity.

[yoavweiss]

:+1: to the slimmer proposal and to speculative

[bizzbyster]

The new proposal requires that all preloads persist at least until the next navigation. And because of this behavior we can get rid of prerender. I like this simplification though it does mean that we are blending what are essentially two orthogonal factors: likelihood that a resource will be needed to render a given page and likelihood that a user will visit a given page next. I'm not completely sure that we should blend these but again the simplification is nice.

But assuming we do it (blend the probability of use on a given page with the probability a user will visit a page subsequently) I'm not sure the keyword 'speculative' or 'low-priority' is even needed. If the probability is less than 1.0 then we can infer that it is 'speculative'. Right?

[igrigorik]

The new proposal requires that all preloads persist at least until the next navigation.

That was the case with previous "preload" hint as well - same policy as all other requests on the page.

I like this simplification though it does mean that we are blending what are essentially two orthogonal factors: likelihood that a resource will be needed to render a given page and likelihood that a user will visit a given page next. I'm not completely sure that we should blend these but again the simplification is nice.

No, you still have two cases: speculative is a flag indicating that you're talking about the conditional probability of resource X given that you navigate to next page; without speculative flag you're talking about probability of using resource X in current navigation context (which could be <1.0).

If we're willing to drop above distinction then yes, we can get rid of "speculative". But, based on the feedback so far (at least from the Chrome eng team) I think it's worth keeping. That said, I'm open to being convinced otherwise :)

[bizzbyster]

"No, you still have two cases: speculative is a flag indicating that you're talking about the conditional probability of resource X given that you navigate to next page; without speculative flag you're talking about probability of using resource X in current navigation context (which could be <1.0)."

Okay! I didn't realize that the speculative flag had this meaning. Speculative strictly means that we are speculating about next page action. Given this, I'm in complete agreement with the proposed logic I'm only just concerned about the confusion caused by this term. So, for instance, if there is no 'speculative' flag and probability is <1.0, I am technically doing something that is in fact speculative. So I think it's just a matter of finding the right term. Why not just say have the flag named "future"?

[igrigorik]

Took a run at updating the spec..

• Currently using "next" keyword for next-page resources.
• Updated + reorganized processing logic.
• Renamed "probability" to "pr"
• Plus lots of other small changes...

It's a sizable rewrite, please take a look and let me know how it looks!

Latest draft: http://w3c.github.io/resource-hints/

[bizzbyster]

This looks good to me. I like the choice of the ³next² keyword. Getting rid of prerender is a really nice simplification IMHO.

Two things occurred to me reading through this that are not exactly on topic:

1. Once again I really feel like we need ³expected-size² to allow the UA to make the right decision about speculative hints. But as you¹ve mentioned this may not be the best place to introduce that.
2. Redirects used to anonymize the referrer before sending the user to the final destination are really stupid ‹ resulting in unnecessarily slower web pages. Is it possible to add a ³no-referrer² attribute to linkable elements so that the web developer can simply inform the browser that it does not want the referrer header to be included in the request? Thanks,

Peter

[igrigorik]

@bizzbyster thanks for the review + feedback :)

Re, 1: as we said earlier, expected size is not specific to RH. Assuming we spec + implement, RH would inherit it (alongside other elements).

Re, 2: for reference, respective what-wg thread @ http://lists.w3.org/Archives/Public/public-whatwg-archive/2014Oct/0090.html