Updating stored headers

[mnot]

7234 requires stored headers to be updated upon a 304 or HEAD response.

However, there are a number of issues:

• It doesn't specify that H1 connection-related headers should be omitted
• Some browsers omit other headers (e.g., x-content-*, content-* and x-webkit-* - see Chrome and Webkit
• Apache has a whitelist of headers; see bug
• Intermediaries are reluctant to do this for performance reasons; e.g., squid

It seems like a lot of the bugs referenced above originated from the confusing language around omitting entity headers in RFC2616; that was removed in RFC7234 (see current language).

I think we need to:

• [ ] Specify the list of H1 connection-oriented headers to be omitted
• [ ] Talk about other headers that need to be omitted (coordinating especially with browsers, who seem to have weird behaviour here)
• [ ] Contemplate any advice/warnings/requirements we can give to an intermediary cache about how they should handle this if they don't want the performance hit

Mozilla's list of headers they don't update might be a good starting point for the first two (see also related bug).

There are also browser tests and an outstanding PR.

Note that this is a security issue, because some browsers filter out headers starting with Content-, which includes Content-Security-Policy -- i.e., a browser that has an old copy of the response won't see the new CSP header on a 304 response.

(this might also apply to 206 responses, since there's header combination there too)

[mikewest]

So sending cookies hooks in above the cache layer, and setting them hooks in above it, in the spec? That seems unfortunate.

Setting the Cookie header happens at step 5.17.1 of https://fetch.spec.whatwg.org/#http-network-or-cache-fetch when the request is being generated, before hitting the cache. That seems reasonable, as we need to know the request's characteristics before consulting the cache to handle Vary, et al.

[MattMenke2]

Yes, I'd argue the setting the cookies in the cookie store should happen at the same layer, however.

[mikewest]

We could move the Set-Cookie processing to the same "HTTP-network-or-cache fetch" algorithm, in the cache miss branch (maybe after step 7.2?). I think that would basically be a no-op, but we'd need to audit direct usage of "HTTP-network fetch". It's not clear to me that that's worth doing... Perhaps @annevk has an opinion?

Either way, it seems like we're handling the problem discussed above in Fetch, not RFC6265bis. Are there still changes we should make to the latter regarding this discussion (short of rewriting the whole thing in terms of Fetch, which I don't think anyone's signing up for)?

[annevk]

I think we could move it and it probably makes sense if that's when they're processed. (That algorithm also takes a credentials flag which I'd hope folks set correctly if they invoke it directly, but I'm not aware of anyone invoking it directly.) We should maybe also update https://fetch.spec.whatwg.org/#http-header-layer-division to account for this somewhat subtle difference.

[mnot]

OK, that seems to take care of Cookies. I'm going to mark the PR as ready for review.

[mnot]

The editors have signed off on #337. Any further comments before we merge?