discussion: new (clear and separate) requirement for CSP reporting/logging?

[elarlang]

Potentially related requirement #1311

Should we have clear separate logging requirement for like (just to give the point, not the finalized wording): Verify that Content Security Policy violations are reported (by browser) and logged (on trusted service layer).

Category: V7.., Level: 2 (CSP ruleset itself with level 1)

[tghosth]

I think this should be a specific requirement as it requires a specific mechanism to catch these logs which are sent by the browser.

Agree with L2.

Does requirement belong with logging or with CSP? I would be inclined to keep it with CSP because it is a different mechanism to the regular logging anyway.

[tghosth]

I still think this should be a separate requirement as it is not a standard log. I would put it as L2 in 7.2 and say:

Verify that a CSP report logging server is in use and that the reports it collects are then logged as part of the application's standard logging mechanism.

What do you think @elarlang

[elarlang]

I think to focus should be different for the requirements. Something like:

Verify that Content Security Policy violations are reported and logged.

We don't need to talk about the server or standard logging mechanism (this is too vague), there are separate services for that and I think it works well that way.

As the Content Security Policy itself is mostly second-layer defense, I think this requirement as one more step is a level 3 requirement.

[tghosth]

Agree with L3

How about:

Verify that CSP violations are reported and logged as part of the application's standard logging mechanism.

[elarlang]

I already shared my opinion on the standard logging part

We don't need to talk about the server or standard logging mechanism (this is too vague), there are separate services for that and I think it works well that way.

What risk do you have when using an external service or what risk are you going to solve, when requiring it to be part of the standard logging mechanism? If I call an external service as my standard logging mechanism, am I ok?

[tghosth]

Discussed with @elarlang that "as part of the application's standard logging mechanism." is more of a recommendation anyway rather than a strict control so it isn't really necessary for here. I will open a PR for this.

Verify that Content Security Policy violations are reported and logged.

@set-reminder 5 mins @tghosth to open PR

[octo-reminder[bot]]

⏰ Reminder Thursday, May 2, 2024 12:38 PM (GMT+02:00)

@tghosth to open PR

[octo-reminder[bot]]

🔔 @tghosth

@tghosth to open PR

[tghosth]

Waiting for #1947

@set-reminder 3 days Josh to address

[octo-reminder[bot]]

⏰ Reminder Sunday, May 5, 2024 12:00 AM (GMT+02:00)

Josh to address

[tghosth]

Addressed in: https://github.com/OWASP/ASVS/commit/6cf791ff8882aea26fb3cdfad7800bb569d69d50

[ryarmst]

Is it too late for community comments on this one?

[elarlang]

Is it too late for community comments on this one?

It's never too late. Just after a release changes need to wait for the next release.

[tghosth]

Hey @ryarmst, like @elarlang says, until the release, it's never too late :)

[ryarmst]

Thanks @elarlang @tghosth, I'll add some thoughts to consider:

1. It was noted that CSP violation logs are fundamentally different than server-side logs as the violation logs are initiated by the client. As a result, log data from the client can be manipulated or prevented entirely regardless of the reporting policy. IMO there is an added risk if developers consider these equivalent to events logged server-side (perhaps also there is a need for additional controls validating these logs compared to standard server-side logs). I have always understood the intent of this reporting mechanism not to detect potential security issues/events, but to ensure that the specified policy properly functions without breaking the page/site. On these grounds, I would question the inclusion of a requirement for CSP violation reporting, which fulfills a usability rather than a security role. If I am an attacker developing a XSS payload, I would develop it in my own environment to ensure it bypasses CSP (therefore not causing a violation) before deploying against a target.
2. I know the requirement does not specify a method, but it's worth noting that the present browser landscape is a little messy. The report-uri directive (widely supported) is technically deprecated in favour of report-to in part as a component of the Reporting API, which is intended to capture additional types of violations (such as from Permissions-Policy). To future proof and generalize the requirement, it may make sense to more broadly require logging of client-side violations supported by the Reporting API. That said...
3. The Reporting API is not yet supported by Firefox and there is a relevant discussion here. Some good points are raised about reporting mechanisms leaking information about the client's environment (such as the use of an extension or configuration that results in a violation), though IMO this is a browser concern rather than the ASVS.

[elarlang]

Hi @ryarmst , thank you for your argumented feedback.

Mostly I agree with the comments, just 2 notes:

I have always understood the intent of this reporting mechanism not to detect potential security issues/events, but to ensure that the specified policy properly functions without breaking the page/site.

For the "report only" purpose exists the Content-Security-Policy-Report-Only solution, when using that, you know when something is not correct from the functionality point of view, but without blocking.

On these grounds, I would question the inclusion of a requirement for CSP violation reporting, which fulfills a usability rather than a security role.

About the functionality - one of the security components is availability, that requires working functionality.

Now, rethinking, is it worth a requirement? Taking my own-written checklist and "Is the application less secure without the requirement", then probably from the security point of view the impact is small and it can be added as a recommendation.

Note, that the proposal was for a level 3 requirement.

ping @tghosth

[jmanico]

What risk do you have when using an external service or what risk are you going to solve, when requiring it to be part of the standard logging mechanism? If I call an external service as my standard logging mechanism, am I ok?

This is mostly to protect against DOS. I can force tons of errors on the client and slam a service with CSP errors. Same with react/angular logging. A third party service to collect CSP errors will radically lessen the load in case of purposeful malicious logging. I prefer not to expose a dedicated logging endpoint in my services.

[octo-reminder[bot]]

🔔 @tghosth

Josh to address

[tghosth]

@ryarmst I think you make some great points.

I have always understood the intent of this reporting mechanism not to detect potential security issues/events, but to ensure that the specified policy properly functions without breaking the page/site. On these grounds, I would question the inclusion of a requirement for CSP violation reporting, which fulfills a usability rather than a security role. If I am an attacker developing a XSS payload, I would develop it in my own environment to ensure it bypasses CSP (therefore not causing a violation) before deploying against a target.

IMHO, it certainly can provide benefits a direct security detection benefit but I agree that an attacker who is serious enough to work on a CSP bypass is probably also serious enough to test without triggering anything.

In principle, it is also useful to detect a broken website as which in theory also counts as security as @elarlang but I think there is a definite question as to whether the benefit is worth the effort.

I decided to see if @scotthelme wants to pitch in as he has done a lot of CSP stuff.

https://twitter.com/JoshCGrossman/status/1787160822856208698

[ScottHelme]

Thanks for the ping @tghosth, I'm happy to add my thoughts.

For over a decade now, I've been running https://report-uri.com because I believe that CSP reporting, and other browser telemetry, is essential. This should also act as my transparency statement that I operate a commercial service that collects CSP reports.

There are only two circumstances when you will receive a CSP violation report from the browser:

1. Your policy is incomplete/incorrect and is blocking something that shouldn't be blocked.
2. You have a genuine violation which is being correctly reported.

The violations that occur in #1 are useful for creating and debugging a policy, this is the method we use in our CSP Wizard tool to help site operators quickly get a first iteration of their CSP ready. You can then refine and iterate on your policy, observing the violations reports as a feedback mechanism, until you are satisfied.

The violations that occur in #2 are then the ones you want to be concerned about. If you have a genuine XSS attacks, or some other issue, that is being reported, I don't feel that depending on the CSP to 'fix' this is the correct approach. The CSP acts as a temporary measure, neutralising the attack in the browser, while you receive the reports, identify the source of the problem and then rectify it in the application itself. Without CSP reporting, this is not possible.

Many people will often refer to a "CSP bypass", in which an attacker finds a way to get script into the page that is allowed to run. This isn't a CSP bypass, but is instead a problem with your CSP not being properly defined, so the reports then fall under scenario #1 above. If the script, or other action, is allowed per the CSP, everything is behaving as it should be.

I don't feel we can have a mechanism like CSP deployed without the feedback that violation reporting offers. CSP has the potential to break functionality on your site, and detect serious attacks. Neither of these scenarios should really be ignored by site operators!

[ryarmst]

Thanks @ScottHelme for the comment and Twitter discussion.

I will first add my overall thoughts on implementing violation reporting as I failed to in my initial comment: I do think reporting is a necessary component of implementing a CSP. IMO the greatest utility comes from implementing a non-breaking CSP, which should be a requirement any time a CSP is implemented.

The violations that occur in #2 are then the ones you want to be concerned about. If you have a genuine XSS attacks, or some other issue, that is being reported, I don't feel that depending on the CSP to 'fix' this is the correct approach. The CSP acts as a temporary measure, neutralising the attack in the browser, while you receive the reports, identify the source of the problem and then rectify it in the application itself. Without CSP reporting, this is not possible.

I don't think anyone has argued that CSP is a 'fix' to XSS, but a robust CSP can mitigate otherwise effective attacks in some scenarios. I would rather generalize it as a defense in depth mechanism rather than a "temporary measure" because there is no point at which a CSP ought to be removed when functioning.

I am skeptical of (but open to) the practical effectiveness of using reporting to identify application vulnerabilities through CSP violations. In any scenario where the attack surface is exposed to an attacker, they can craft payloads without sending reports. A successful attack (looking at XSS still) requires both an application vulnerability AND a CSP policy that will not block at least one viable exploitation of the vulnerability. Therefore, the most impactful vulnerabilities (the ones that can be exploited) will not generate reports. As @jmanico mentioned, an attacker also has the opportunity to pollute an exposed logging endpoint for misdirection or other purposes (though I have not heard of such an attack).

That said, I can think of a few possible scenarios where a report would be more likely to be triggered:

1. An attacker attempting blind XSS against environments outside of their control.
2. Non-targeted exploitation attempts and scanning.
3. Incidental (unintentional) violations through user error or other means.

@ScottHelme also noted (Twitter) that he has seen many cases of reporting being the only identification vector of potential vulnerabilities. I have not seen any writeups or other evidence/data myself, but - based on his claim - I have changed my position in favour of reporting as a proactive security control. To be fair, as an appsec consultant, I mostly work with clients who either have not implemented a CSP or have implemented an overly permissive policy where basic attacks would not result in a violation, so I have very rarely seen situations where an organization would be positioned to make use of violation reports in this way.

Many people will often refer to a "CSP bypass", in which an attacker finds a way to get script into the page that is allowed to run. This isn't a CSP bypass, but is instead a problem with your CSP not being properly defined, so the reports then fall under scenario #1 above. If the script, or other action, is allowed per the CSP, everything is behaving as it should be.

I do not agree with this framing. I think there are enough scenarios where a robust CSP will not block certain vectors. For example, even use of the script-src nonce-... pattern can be defeated if the application is inappropriately embedding attacker-controlled content into a nonce-protected region.

I don't feel we can have a mechanism like CSP deployed without the feedback that violation reporting offers. CSP has the potential to break functionality on your site, and detect serious attacks.

I absolutely agree that reporting must be part of the CSP deployment process. I am still unsure of the practical efficacy for "detecting serious attacks" but I will defer to @ScottHelme's experience.

[elarlang]

It is defense in depth, it is useful.

Now the question still (https://github.com/OWASP/ASVS/issues/1445#issuecomment-2092418039) is, should we use it as a level 3 requirement or as a recommendation?

[tghosth]

At this point, I am leaning towards a relatively non-prescriptive L3 requirement. Whilst the definitions are not yet finalized, L3 is always a "stretch goal" and so I don't think this puts an unnecessary burden or risk on implementers.

Current wording being proposed:

#	Description	L1	L2	L3	CWE
7.2.7	[ADDED] Verify that Content Security Policy violation reports are logged.			✓

[elarlang]

I prefer:

Verify that Content Security Policy violations are reported and logged.

For the chapter, I prefer "V50.2 Browser Security Mechanism Headers"

... and I'm also ok, when we cover it as a recommendation.

[tghosth]

I see this as logging and would prefer to keep it here, especially so as not to overload V50

[elarlang]

I know I opened the initial proposal for V7, but here has been a nice amount of feedback and arguments to rethink it all. I don't think we should focus on logging with this one.

Logging is the result of the configuration. If you don't have this in place, you don't have logging.

On the application side you need to make a configuration to have the logging somewhere. This somewhere can be an external service and out of the scope of the application.

We are not going to verify "logs exist on external service" but "Verify that CSP declaration has reporting defined".

Also, if you just require logging, it contains a hidden requirement to have the correct reporting declaration for CSP in place.

[tghosth]

Yeah but the hard part here isn't adding a bit of text to the CSP header, it is having the infra to collect the violation logs which is why I think it makes more sense here.

[elarlang]

The solution for receiving the reports can be whatever and I would not dictate it, it can be external service and out of scope, therefore.

[tghosth]

Yeah but it is a logging thing more than it is a browser header thing

[tghosth]

I discussed this with Elar and he made two very good points

1) The report logging mechanism almost certainly will be outside of the application and is therefore technically not in scope for ASVS as it isn't really up to the developer to maintain it. 2) From a verification perspective, we are more likely to verify the HTTP headers rather than the existence of a logging server.

As such, I am comfortable moving this to V50.

I would suggest:

[ADDED] Verify that the Content Security Policy header specifies a location to report violations.

[elarlang]

ping @ryarmst @ScottHelme - any more comments on the latest proposal? If not, I'll go for PR.

[ryarmst]

@elarlang looks good to me!

[elarlang]

I made the PR, not that I used the header name (as all other requirements in the section do), not the policy name. Also modified 50.2.1. https://github.com/OWASP/ASVS/pull/1957/commits/f08863c10a9f689455867bcef454161429fea095

[tghosth]

@elarlang do you want to remove this from 7.2.7 as well?

[elarlang]

@elarlang do you want to remove this from 7.2.7 as well?

Yes, thanks. I added it as a new requirement and did not recheck/remember, that it already got into the repo in another place.