CORS-safelisted request-headers and Client Hints

[mnot]

Whitelisting request headers creates a perverse incentive for applications to use query strings rather than request headers for cross-site requests, to avoid preflights.

For example, there's an emerging specification (no public link, sorry) for media players to send hints to the origin. They define a request header, but because of CORS, also define a query parameter to convey this information in.

This doesn't seem like a good way to go; not only is the query string not intended for colonisation by standards documents, it's also going to make caching and other generic functions more difficult to interpose, and less efficient. Linking can become problematic too.

It also seems like Client Hints are going to make this worse, as they get defined. While browser-defined hints will get included in the whitelist naturally, anything defined by other parties (whether it's the site in question, a common service, or something like video clients using fetch) are going to suffer.

The problem is the artificial pressure that CORS places on people who want to send such information to the server; it discourages using headers and encourages using query strings (or other parts of the URL).

Possible solutions (in rough preference order):

1. Turn the whitelist into a blacklist. AIUI the threat model that the whitelist is trying to mitigate is headers that the server may rely upon for security-critical processing; especially, Content-Type. Could we re-evaluate whether the whitelist is the most appropriate way to do this?

2. Make it easy to submit additions to the whitelist, and publicly commit to a SLA on processing them (or at least an intention to expedite them). In the example above, we could ask the folks defining the emerging spec to do that to make using the header more viable.

3. Define a prefix for CORS-whitelisted request headers that opts them out of the whitelist -- the idea being that the prefix will effectively block any existing header in use, so any header that uses it will be processed with the knowledge that it's exposed to cross-site client code (sort of like a reverse of Sec-).

/cc @igrigorik @yoavweiss @annevk

[yoavweiss]

Can you detail how Client Hints are making this worse?

[annevk]

@martinthomson and I have been musing on a TLS connection flag that opts out of all CORS preflights for the connection. That coupled with sufficient pushback against non-header-based approaches when headers are appropriate could be a viable alternative.

As for your suggested solutions:

1. This seems rather scary. The threat is really being able to trick the server and it's unknown what all servers can be tricked with. What is known is that since the dawn of HTML forms they had to at least not be susceptible to them.
2. This would require servers to pay attention, forever. (Or get a better architecture, if they haven't already.) To some extent they have to pay attention because of TLS, but also not really.
3. This is a variant of 2 with a one-time cost.

Unfortunately, none of them deal with the fact that these days there's not just restrictions on headers, but also the size. We restrict the maximum length of a Referer header (currently 4096 bytes, but 2048 is/was investigated) and each header that can bypass a CORS preflight can have a value that's a maximum of 128 bytes (and there's a further maximum of 1024 total, but that cannot be reached). This helps prevent information leaks and certain cache attacks.

I suspect that if we opened the floodgates, everyone would start hitting the size limits pretty quickly and we'd be back at square one.

[yutakahirano]

Can you tell me why they hate CORS preflight? Is that for the additional round-trip, or the server implementation cost?

[martinthomson]

From my experience there are two simple cases in which you might care about preflights. In the first, you are happy that they exist, because you are unsure that you have properly hardened your service against all potential accidents. Those services are why CORS exists, but I always recommend against anything that relies on this (weak) protection.

In the second, you build the service with a strong authorization model that doesn't rely on ambient authority. Having to build code to answer OPTIONS requests is annoying, in addition to the additional round trip that you end up paying. I've found that there is a tendency to also want to limit positive responses to only those responses that work and those header fields that they know are good. That leads to a variation on this issue, where the CORS policy tends to block new stuff, and you have to negotiate an exclusion. That's down to what your internal processes look like, but it can take time; I mean, it's not negotiating a change to this spec, but it isn't free either, especially when listing all the header fields starts to take more and more bytes.

[LPardue]

@yutakahirano said:

Can you tell me why they hate CORS preflight? Is that for the additional round-trip, or the server implementation cost?

Video streaming is moving towards solving low-latency streaming challenges. Conventionally, a stream consists of a sequence of segments and each video segment has a URL from where it can be retrieved. Low-latency designs include shorter segment durations, or intra-segment fetches (sub-URLs for want of a better description). Both approaches increase the frequency of requests by a varying degree. Some deployments feel the burden of increased requests per second, be it the cost to serve a request, or the additional overhead of request logging etc.

So in the face of already having to handle more requests/second to solve a streaming technical challenge, the requirement to also double that due to CORS becomes troublesome.

[mnot]

@martinthomson I suspect that Origin Policy would work as well; while you pay a RTT for the first request, it's cacheable and origin-wide (unlike OPTIONS). For video, that's probably wearable. On h2+, you could even do a parallel OP request to the initial OPTIONS, possibly triggered by an OPTIONS response header.

Or is there another reason you're considering a TLS extension (which is much harder to deploy - but maybe that's what you're wanting)?

@annevk your response to my suggestions leaves me wondering about the threat model and the browsers' place in addressing it. A server that changes its behaviour based upon a new header that it defines takes on the responsibility for assuring that it's not vulnerable; browsers aren't assumed to protect them from everything. CORS was put in place because of vulnerabilities exposed at scale by XHR and friends, it's not a general solution to securing servers.

To put it another way - a server can be tricked if the URL query string contains data that it doesn't anticipate, and yet browsers have gone off and allowed any old query string in requests. How could you?

[martinthomson]

The harder to deploy thing is almost certainly a feature, for sure. I'd say that once software is in place, flipping the switch should be easy, it's the process you might want to follow before you flip the switch that is challenging.

The other concern I've heard is that preflighting (or fetching Origin Policy) is expensive, and a flag in the connection setup would avoid that cost.

Origin Policy has the disadvantage of needing to define a scope over which it applies, with interesting characteristics from a privacy perspective as a result. On the other hand, it's fairly clear what the scope of applicability for a TLS flag is.

[mnot]

The harder to deploy thing is almost certainly a feature, for sure. I'd say that once software is in place, flipping the switch should be easy, it's the process you might want to follow before you flip the switch that is challenging.

From a server perspective, it's another weird switch to explain and support.

And, given that the people who operate these things are usually very separated from the people who actually write the applications, I suspect you're going to fail miserably at making things actually more secure; the flag will either be turned on by default, or by a "helpful" administrator who doesn't have a clue about whether or not it's actually safe to do so.

The other concern I've heard is that preflighting (or fetching Origin Policy) is expensive, and a flag in the connection setup would avoid that cost.

Preflighting is expensive, but if you amortise it across many requests with a cache, it becomes much less onerous.

Origin Policy has the disadvantage of needing to define a scope over which it applies, with interesting characteristics from a privacy perspective as a result. On the other hand, it's fairly clear what the scope of applicability for a TLS flag is.

I thought it was the origin...

[martinthomson]

I thought it was the origin...

Over what period?

A flag on the connection could be considered to apply to all requests made on that connection. I don't think that you can say anything so concrete about an Origin Policy.

[annevk]

@mnot the same-origin policy was put in place to protect user data on servers. When it was put in place it had a number of enshrined "weak points", such as allowing arbitrary GETs without custom request headers. When CORS was put in place the same-origin policy was "weakened" ever so slightly and that has caused a bunch of collateral damage. Overall it was probably good in that it allowed many more interesting things, but we're still fixing the finer details and hearing about security vulnerabilities (both browser and server side). So yeah, proposals to "weaken" the same-origin policy further will continue to have a tough time and have to be balanced somehow.

(The same-origin policy being what works if the server does not opt into CORS. CORS is really about opting out of restrictions, not about restricting.)

[LPardue]

I might be missing something but it seems part of the origin policy discussion here actually seems to overlap with the Client Hints Reliability proposal - https://datatracker.ietf.org/doc/html/draft-davidben-http-client-hint-reliability

[annevk]

Indeed, see also WICG/client-hints-infrastructure#100. Sending a bunch of new headers cross-origin still seems rather problematic to me. The reliability proposal may or may not address that. (It's not clear if the plan is to use that for client hints in general.)