bugcrowd / vulnerability-rating-taxonomy

Bugcrowd’s baseline priority ratings for common security vulnerabilities
https://bugcrowd.com/vrt
Apache License 2.0
443 stars 84 forks source link

Add Flash-based Cross-Site Scripting (XSS) as P4 #120

Closed EgiX closed 6 years ago

EgiX commented 6 years ago

At the moment there's no difference between Flash-based Reflected XSS and classic Reflected XSS, they're all considered P3 bugs according to the current VRT version.

IMHO, considering Flash-based XSS as P3 is a bit overrated:

https://www.theverge.com/2015/12/1/9827778/stop-using-flash https://techcrunch.com/2017/07/25/get-ready-to-say-goodbye-to-flash-in-2020

For these reasons I believe we should have a new VRT category for Flash-based Reflected XSS, handling them as P4 (and maybe downgrade to P5 after Flash EOL in 2020?), just like e.g. XSS > IE-Only > Older Version (IE 10/11).

What do you think about it?

truemongo commented 6 years ago

+1 on a new VRT entry for Flash-based XSS; this attack is increasingly less practical. I agree with the move to P4, and the possible further downgrade when Flash goes EOL.

EdisK commented 6 years ago

I agree on having a specific P4 entry for Flash-Based XSS given the prerequisites (enabling flash, clicking on the screen to get pop up). Let's make this happen. Thanks, @EgiX!

trimkadriu commented 6 years ago

+1 on adding this new entry!

chrashley- commented 6 years ago

+1 to add this as well

plr0man commented 6 years ago

Some context on why there's no foolproof solution here: https://github.com/bugcrowd/vulnerability-rating-taxonomy/pull/72

ghost commented 6 years ago

This proposal is interesting but probably premature. Unfortunately, Flash is still being used actively and developers seem to ignore the benefits of switching to HTML5. Believe it or not, people will enable Flash out of necessity or disregard and right now, its usage is around 17%.

It is also important to point out that Flash won't be receiving updates after 2020 and until then, it can't be considered deprecated. It is probably wise to wait a bit longer to dismiss this threat and then consider this type of issue as P4.

plr0man commented 6 years ago

Closing this issue as discussed during today's Vulnerability Roundtable

truemongo commented 6 years ago

Since I just referenced this issue, I thought I'd also comment here on @plr0man's reference to #72 and specifically the rationale for not downgrading Flash-based vulns:

If we were to follow this logic, every class of vulnerabilities that could potentially be found in Flash would have to get its "Flash-Based" entry downgrading the base priority by one point.

I don't think that is necessarily true. For example, open redirects are already P4 (just on the edge of "best practice"), so I don't think it would be mandatory to downgrade it further. Reflected XSS, however, is P3, a higher priority level where relative impact matters more. Having a Flash-based RXSS marked with the same severity as a non-Flash XSS, at this point, does not accurately reflect their relative impact / practical use.

EgiX commented 6 years ago

@truemongo That's exactly what I've tried to explain when I've opened the issue; the point is not just having Flash to be considered deprecated (and insecure), but the amount of user interaction required:

Reflected XSS is P3 and you (as victim) usually need just one click to get the payload executed. With Flash-based XSS you (as victim) must have Adobe Flash Player installed on your system, click on a malicious link, then click "Enable Flash Player for this site", and only then payload will be executed (sometimes you also need to click some button to trigger the payload). For this reason I think the exploitation likelihood is a bit lower than a vanilla Reflected XSS, as such they shouldn't have the same priority/risk.

EgiX commented 6 years ago

Furthermore, I'd also like to comment on @rsmith31415's note:

This proposal is interesting but probably premature. Unfortunately, Flash is still being used actively and developers seem to ignore the benefits of switching to HTML5. Believe it or not, people will enable Flash out of necessity or disregard and right now, its usage is around 17%.

Where did you see Flash usage was around 17% in January? According to W3Techs in January it was 5.5% and right now is 5.2% (down from 7.2% over a year ago).

As such, I don't think my proposal was premature: if the trend continue to decrease at this rate, it would probably make sense to have all "Flash-based" issues to be downgraded by one point and further downgrade to P5 even before 2020 (in case the trend goes to <0.1% before Flash EOL).

Please have a look at this page to further see how Flash usage is drastically decreasing .

ghost commented 6 years ago

@EdisK The source was Flash Usage Trends (which is the same link you referenced below) and this number was reported when Adobe announced that it will stop supporting Flash in 2020. I think 5.5% refers to usage as programming language which looks like a different metric.

plr0man commented 6 years ago

Thanks everyone for input here. If we were to go down this path it looks like this would be the list of entries qualifying for downgraded Flash-based variants: external_authentication_injection cross_site_scripting_xss open_redirect

As far as I can tell other potentially affected entries with the exception of application wide CSRF don't have baseline priorities. To accommodate that, we would be enforcing general Flash-based downgrades and adding a paragraph in our Standard Disclosure Terms. Does that sound like are reasonable solution?

ghost commented 6 years ago

I think @truemongo already pointed out that it is not a good idea to downgrade Flash-based vulnerabilities in general and instead, they should be considered on a case by case basis. Furthermore, a moral rigorous process is needed to assess when a vulnerability is not a significant risk anymore (you can set an arbitrary threshold if you want).

plr0man commented 6 years ago

As far as I understand this statement was not intended to be a red flag, but an option:

I don't think it would be mandatory to downgrade it further

If using Flash is a prerequisite significantly lowering risk, it would be an equal factor for all issues. For example if there's a Flash-based CSRF that leads to account takeover, this should now mean one priority point less, so instead of the average P2 in such cases it would become a baseline of P3.

While there's a point to not necessarily adding a variant for the existing P4 entries, because there's still going to be some minimal risk, it is important that we are consistent. We give our customers the option to redefine what they view as the minimums via the brief or by accepting Informational reports

ghost commented 6 years ago

Sure, but there are vulnerabilities already classified as P4 and wouldn't be appropriate to classify them as P5 if you apply that particular rule. Furthermore, I think the risk of Flash-based vulnerabilities is not low enough to warrant a downgrade. That is the reason I suggested a more rigorous process to determine when a vulnerability doesn't pose the same risk as before.

plr0man commented 6 years ago

Sure, but there are vulnerabilities already classified as P4 and wouldn't be appropriate to classify them as P5 if you apply that particular rule.

Can you elaborate, do you have any specific entries in mind?

I think the risk of Flash-based vulnerabilities is not low enough to warrant a downgrade.

That is the decision we should make first. It looks like most of us do agree that Flash-based should warrant a downgrade. But feel free to provide counterarguments if you believe you can change this opinion.

That is the reason I suggested a more rigorous process to determine when a vulnerability doesn't pose the same risk as before.

If we agree that RXSS which in many cases is a very strong P3 (hooking up to the browser, stealing CSRF tokens, taking over account etc.) should be downgraded to P4 if it's Flash-based, then what rigorous rules would you see that we could apply to other entries?

ghost commented 6 years ago

I was thinking about open redirects specifically, although some types of "Same Origin Method Execution" could qualify as P4.

That is the decision we should make first. It looks like most of us do agree that Flash-based should warrant a downgrade. But feel free to provide counterarguments if you believe you can change this opinion.

Sure. Actually, you provided good counter arguments here:

We were able to discuss internally the issue you raise here. While you are making a valid point, the proposed change does not seem to be a good option at this time. For one, other than open redirects there is a multitude of other issues that could occur in Flash (XSS, CORS issues, content spoofing etc.). If we were to follow this logic, every class of vulnerabilities that could potentially be found in Flash would have to get its "Flash-Based" entry downgrading the base priority by one point.

We strive to keep the VRT target agnostic and while there might be limitations to certain PoCs we feel that reserving the right to downgrade (and upgrade) at ASE-customer discretion is the best solution in those cases.

Flash-based vulnerabilities certainly need several prerequisites (install and enable Flash), but we often forget that regular users are prone to installing and enabling Flash without much thought and even more tech-savvy users are forced to enable Flash for certain sites. As a concrete example of the pervasiveness of this practice, the federal government site used to file tax returns in my country requires Flash, so I think the idea that Flash-based vulnerabilities are no longer practical or likely is actually motivated by individual circumstances. Instead, we should judge objectively whether such vulnerabilities are a sizable risk or not.

By the way, I'm not opposed to downgrading specific types of Flash-based vulnerabilities based on their merits, but a more measured approach is better here.

If we agree that RXSS which in many cases is a very strong P3 (hooking up to the browser, stealing CSRF tokens, taking over account etc.) should be downgraded to P4 if it's Flash-based, then what rigorous rules would you see that we could apply to other entries?

I was trying to say that rigorous rules must be applied to determine when a vulnerability can be considered as low risk. I suggested to use a threshold based on the percentage of users that are exposed to such vulnerabilities, so in this case, a proxy of that risk would be the overall usage of Flash. If you arbitrarily set that threshold to 2% or 5% (or any other reasonable value), then you wait until that value is reached and then you downgrade this class of vulnerability.

plr0man commented 6 years ago

The arguments provided in #72 are still valid, but I feel like we came up with a potential solution:

If we were to go down this path it looks like this would be the list of entries qualifying for downgraded Flash-based variants: external_authentication_injection cross_site_scripting_xss open_redirect As far as I can tell other potentially affected entries with the exception of application wide CSRF don't have baseline priorities. To accommodate that, we would be enforcing general Flash-based downgrades and adding a paragraph in our Standard Disclosure Terms. Does that sound like are reasonable solution?

We can always add downgrades for any other VRT entries that we missed here at a later time, but do feel free to name any that I might have missed.

By the way, I'm not opposed to downgrading specific types of Flash-based vulnerabilities based on their merits, but a more measured approach is better here.

It seems like you do agree with a Flash-based downgrade for XSS, but at the same time mention that there should be more rigorous rules in case of other entries. I could agree with this argument if RXSS had a weak baseline of P3, and any easy prerequisite would be just enough to swing it to P4, but it is not the case, quite the contrary. That being said if we agree that RXSS doesn't defend its P3 when Flash-based then this would be our most rigorous test for a general prerequisite.

Regarding the decision as far as general Flash-based downgrade goes, that is still up for discussion and thank you for your input @rsmith31415, you're giving some interesting examples. Let's wait for more feedback from others

shpendk commented 6 years ago

I'm fine if we start by downgrading these without a general rule to downgrade flash-based vulnerabilities:

external_authentication_injection
cross_site_scripting_xss
open_redirect

Though I do think that even with a 17% usage, the requirement of flash downgrades the risk because it only affects a subset of potential targets, and the barrier for enabling/using flash has grown bigger with time.

ghost commented 6 years ago

It seems like you do agree with a Flash-based downgrade for XSS, but at the same time mention that there should be more rigorous rules in case of other entries.

Well, I said I'm not opposed to a downgrade if there is evidence that the risk for that vulnerability has decreased sufficiently. However, so far the arguments in favor of downgrading seem to be based on a merely subjective impression that Flash-based vulnerabilities are not practical or viable anymore. That's the reason I'm suggesting that decisions of this type should be made taking into account objective measures of risk.

As a general rule (statistically speaking), 17% is not nearly enough the percentage deemed to be "acceptable" risk. Of course, this number varies across industries, but the percentiles are often 5%, 2.5%, 2%, 1%, etc. To give some context, the web browser market share for Safari and Firefox is around 15% and 8%, respectively. It would be inappropriate to downgrade vulnerabilities affecting Safari or Firefox because both are below 17% in market share.

plr0man commented 6 years ago

There is seems to be no reliable source for Flash usage metrics directly relevant to our discussion:

What we know for a fact is that browsers don't make it easy for users to use Flash and looking near term this will only get stricter. We are open to more opinions and since this is not a question of if but when, we are essentially looking at two options:

  1. Implement the proposed changes in VRT 1.4
  2. Implement the proposed changes in VRT 1.5
ghost commented 6 years ago

Correct. I think it would be appropriate to change the priority when convincing evidence becomes available that the percentage of users has reached a threshold, but that threshold should be decided now. I suggest 2.5% since you seem very eager to downgrade and most measures of acceptable risk use ~2%, although I wouldn't use VRT releases as a parameter to decide when to downgrade. If a study becomes available tomorrow, then you should downgrade tomorrow.

plr0man commented 6 years ago

The purpose of this issue is not to classify common vulnerabilities that rely on user enabling Flash as accepted risk. If that was the case we would be proposing that Flash-based app-wide CSRF would be P5 not P2, RXSS would be P5, not P3 etc. This will be the case down the road, but not yet.

What we are considering here is risk limitations as a direct result of browsers limiting Flash usage, which results in users needing to take additional steps as a prerequisite for a successful attack vector. That being said, given our research, experience and the feedback provided in this issue a decision has been made to include the proposed changes as part of the VRT 1.5 release

ghost commented 6 years ago

I wasn't referring to "accepted risk" as an equivalent to P5, but as a general term often used in risk assessment, outlier analysis, etc.

I still can't see the value of adding this downgrade to a schedule. This is not a decision that requires a judgment call because there is a better way to objectively measure risk.

bilbomal commented 6 years ago

90% or more users have Adobe Flash Player installed on their systems. But security risk is more than Open Redirect, because Open Redirect don't execute JS code on the victim server.

bilbomal commented 6 years ago

But most importantly, I believe Flash can now be considered kinda "deprecated".... But most importantly, I believe SQLi can now be considered kinda "deprecated" But most importantly, I believe RXSS can now be considered kinda "deprecated" But most importantly, I believe RCE can now be considered kinda "deprecated".

You live in real word and must know that most of "deprecated" we can see in nowadays. I think that Bugcrowd made a rash step with this change.