Issues - Being able to adjust specific issue sample rate for Content Security Policy errors

[Angelodaniel]

Problem Statement

In this case the Content Security Policy errors eat up most quota but are still important to the developers In order to reduce the amount of traffic and save some quota I want to know if it's possible to use something similar like the tracesSampler for JS however it seems that there is no similar option for CSP (Sentry Docs).

Also the Sentry UI doesn't give options to do like a Dynamic Sample on all CSP issues so it's either filter them out completely or don't filter and accept all.

Solution Brainstorm

No response

Product Area

Issues

┆Issue is synchronized with this Jira Improvement by Unito

[getsantry[bot]]

Assigning to @getsentry/support for routing ⏲️

[getsantry[bot]]

Routing to @getsentry/product-owners-issues for triage ⏲️

[lobsterkatie]

The reason there's isn't a tracesSampler-type option for CSP reports is because they're not sent by the SDK, they're sent by the browser.

It seems like it might be possible to have the SDK listen for and catch them (and therefore possibly be able to sample them - would depend on whether the SDK can prevent them from being sent or only observe them on their way out the door), but only in cases where the SDK is already initialized when the violation occurs.

We could also set up sampling and/or rate limiting on our end, but I suspect that would have to be a larger conversation. I'm therefore going to transfer this to the JS repo, as I think it's your best bet as at least a place to start.

[lobsterkatie]

Also of possible interest that I just found: You're apparently not the only one with this request, though the conversation seems to have died out in 2019. Perhaps you could revive it!

[AbhiPrasad]

It seems like it might be possible to have the SDK listen for and catch them

I think this is entirely new behavior for CSP functionality as we would have to build a new integration to the SDK (and add new envelope logic) to make this work. IMO we should be working toward adding some controls in the UI to help with this.

I'm not even sure which team should own CSP stuff - I'll defer to @smeubank re: this.

[getsantry[bot]]

Assigning to @getsentry/support for routing ⏲️

[ryanseddon]

Did anything come of this? I'm interested in sampling CSP too but unsure if that's feasible due to the browser controlling sending of reports.

[smeubank]

after chatting with @oioki this is something which happens outside of the JS SDK. If we want to to filter we need to investigate something with inbound filters potentially. I at least would vote to come up with a way to drop these on server side, project config. And not add this complexity to the SDK just to sample.

cc @rachrwang

[getsantry[bot]]

Routing to @getsentry/product-owners-issues for triage ⏲️

[malwilley]

Thanks for getting to the bottom of this! Will place on the backlog, but this isn't something that's on our roadmap currently.

[scottohara]

Understand and appreciate that this request has been moved to a backlog, and I'm very much looking forward to one day being able to reduce some of the noise from CSP reporting that is drowning out the "signal" from genuine application-level errors.

I do wonder if there's a potential "quick win" here, by tapping into the existing "Filter out errors known to be caused by browser extensions" feature?

Currently, I have this filter turned on (and I assume it's working, as I don't see too many application-level errors that could be attributed to browser extensions); however I do still quite regularly see CSP reports such as:

blocked_uri         wasm-eval
disposition         enforce
effective_directive script-src
source_file         chrome-extension
violated_directive  script-src

My view is that if someone has opted into filtering out errors caused by browser extensions, that should extend to CSP reports as well?

[jangjodi]

Hi @scottohara, thank you for the suggestion. This is a valid request, but given our current bandwidth it's not something we’ll likely get to soon.

[Radiergummi]

I would like to add a +1 here too; we use a misbehaving chat plugin on our site, and CSP reports for that plugin have just accumulated to 83k reports. Not only does this start to threaten our quota, it's also (as Scott said) drowning out other issues.

I understand this is causing headaches with regards to the business model, but I would advocate for an option to drop report requests on the Sentry side without incurring charges. CSP violations quickly stagger up, and the only sensible thing for us to do is turn them off entirely to avoid spending our budget on pointless issues, which cannot be the right way to solve this.
Now that I think of it, this would even allow for a possible attack scenario: If an attacker figures out a CSP report URI points to Sentry, they can first cause millions of CSP violations until the victim's quota must be exceeded, and then work undisturbed while Sentry is "blinded" until the billing period concedes. Sure this is a little contrived, but still imaginable.

[scottohara]

We have reached the point where we’re considering moving our CSP reporting away from Sentry, due to the fact that CSP errors are treated the same as application errors and both come out of the same Sentry quota.

I won’t relitigate the arguments for allowing filtering or sampling of CSP errors as they are detailed above, but I do think that they should be given a separate usage quota so that an influx of CSP errors won’t stop ALL errors from being monitored.

There is a fundamental difference between application errors and CSP errors, in that application errors are (generally speaking) actionable by the developer. Fixing a bug, improving validation or refactoring code to be more defensive are all options to reduce the number of errors reported. The aspirational (but probably unachievable) goal is zero errors.

But CSP errors are not actionable, as they generally represent attempts at doing something that the developer has strictly forbidden by policy. Once a policy has been fine tuned to only allow the exact sources and all false positives have been addressed, there is nothing further the developer can do. There is no aspirational target of zero CSP errors. CSP errors are to be expected.

Having these two different classes of errors come out the same Sentry quota makes it impractical for us to continue with CSP reporting via Sentry. We hope that in future this changes.

[leeandher]

@scottohara @Radiergummi Thank you for your responses and insights. I can definitely see the issue and agree that this would be a useful addition. The comments and responses in this thread continue to build a case for getting this re-prioritized and the team will continue to keep tabs on this thread as we figure out a way forward, but unfortunately it's still not something we can accommodate at the moment.

[scottohara]

For the benefit of others looking for a solution to filtering CSP noise from Sentry without either

1. relaxing your CSP strictness, or
2. turning off CSP reporting entirely

...we are exploring a solution that uses a combination of Content-Security-Policy and Content-Security-Policy-Report-Only headers to maintain a strict policy, but at the same time control which violations will be reported to Sentry.

For example, one of the noisiest CSP violations that we see is Blocked 'font' from 'data:'. That is:

effective_directive  font-src
blocked_uri          data

One of the biggest culprits for these violations seems to be browser extensions. For example, the Grammarly browser extension is one that we see commonly in traces where this violation occurred.

The relevant part of our policy currently looks something like this:

Content-Security-Policy: font-src 'self' https://fonts.gstatic.com; report-uri <sentry uri here>

(Note that data: is not a permitted font-src, nor do we wish to allow it.)

Ideally, we want to continue blocking attempts to load fonts from data URIs, but equally given the volume of such violations, we want to stop sending them to Sentry so that they don't consume our entire error quote (accepting that, in doing so, we lose visibility of these violations).

To achieve this, we can change our policy headers as follows:

- Content-Security-Policy: font-src 'self' https://fonts.gstatic.com; report-uri <sentry uri here>
+ Content-Security-Policy: font-src 'self' https://fonts.gstatic.com;
+ Content-Security-Policy-Report-Only: font-src 'self' https://fonts.gstatic.com data:; report-uri <sentry uri here>

First, remove report-uri from your actual policy. Everything else remains the same, so you're still blocking the same violations, but nothing will be reported to Sentry (yet).

Next, add a new Content-Security-Policy-Report-Only header that is exactly the same as your actual policy, but also:

• includes report-uri for reporting violations to Sentry
• includes data: in the font-src directive (and any others that you want to exclude from Sentry)

I hope this helps anyone in the same situation, struggling with Sentry's decision to combine application-level errors and CSP errors into a single quota.