Keying on Vary:ing headers in prefetch/prerender

[domenic]

Spinning off of https://github.com/WICG/nav-speculation/issues/18, but more general:

We currently key prefetches and prerenders on (referrer policy, URL) in the spec. (Prefetches might also key on sandbox flags present? Can't really tell if that's a key in the same sense...)

18 discusses removing the referrer policy key.

However we should probably also key on any headers that appear in the Vary response header.

This is a bit tricky as it will involve essentially doing the header computation both at original prefetch/prerender time, and at activation time. The fetch spec isn't well layered to allow this, as it assembles all the headers during the main fetch algorithm. (Maybe we could hack it with an only-if-cached fetch?? But that's only for same-origin, and wouldn't really match implementations anyway...) But it seems like the right thing to do.

[jeremyroman]

Agreed that this can be tricky incl in the presence of redirects. If it isn't already, maybe the logic that computes the referrer could be factored out so that it can be reused to do check Vary: Referer explicitly?

Or do we want to consider Vary for other request headers? It's kind of contrived to consider other cases where the request headers could vary, but in principle do a whole "dummy" fetch where we intercept all the network fetches and examine them. As you say, Fetch isn't currently well-factored to allow this to be done in a lucid way.

[domenic]

I was hoping to consider Vary for other request headers, as that seems more semantically desired. However, upon reading articles like https://www.fastly.com/blog/best-practices-using-vary-header and https://www.smashingmagazine.com/2017/11/understanding-vary-header/ it seems like referer is one of the few common and legitimate use cases that would come up here. (Others that people commonly use are Accept and its friends, or User-Agent, but those shouldn't come up here.)

So indeed, maybe we can just do the referrer calculation, conditional on Vary: referer.

I admit not having thought much about redirects.

[domenic]

Some minor updates:

• We no longer key off referrer policy in the spec.
• Above I said User-Agent should not come up. In Chromium we've found it actually does come up when users on mobile browsers use the "view desktop site" toggle. So, if Vary: User-Agent is present, that should definitely cause a cache miss.

[domenic]

Another good use case for Vary is varying on client hints. E.g. Vary: sec-ch-prefers-color-scheme implies your server will generate different responses for dark/light mode.

[jeremyroman]

Interesting. I'd have hoped/assumed that light/dark mode adaptation would be almost exclusively done client-side, with media queries (CSS, <picture>, and whatnot). If only so that the site can react when the user changes between the two without a reload.