Verification of Findings - Reducing the number of false positives for vulns that open source projects receive

[fcanogab]

Open source maintainers usually receive a lot of feedback about vulnerabilities found by security tools in open source projects. Sometimes, the quality of the information provided is not very good, and it generates a lot of work for the open source maintainer that maybe could be avoided.

I would like to propose to work, maybe within this working group, on this issue.

I'm thinking about things like:

• Publishing a checklist of things to do before reporting. What you have to check before sending a finding from a security tool to an open source project maintainer. For example, verify it "manually".
• Develop guidelines about how to manually verify some security findings usually reported by tools (e.g. security headers, root cause analysis...).
• Publishing a template for reporting potential flaws. What it is interesting that you include in your report.
• Improvements of security tooling. What developers of security tools can do to improve the information of their findings.
• Talking about backporting and why the package versions cannot be the only evidence of the existence of a flaw.
• Publishing examples, statistics...
• Others...

I'm not saying to create anything new, if something exists, but work on whatever it takes to:

• Reduce the number of false positives about vulnerabilities that open source maintainers receive.
• Increase the quality of the information that open source maintainers receive when someone reports a potential vulnerability to them (a patch > a reproducer >>>>>>>> a dump of a scanning tool).

1- Do you think that the problem exists and the project is worth it or not? 2- Do you think if it is a project that may fit in this project?

[MarcinHoppe]

I very much think the project exists. At least the first 3 bullets are something that we discussed in a few WG meetings as an component of a good security policy for OSS projects.

I am a little less clear on how we might improve it on the tooling side. @fcanogab can you give us a bit more details on what you had in mind?

[fcanogab]

Not sure Marcin, it's just an idea. Maybe some tools are reporting vulnerabilities just based on package versions, thus reporting wrong information (due to backporting for example). People that develop scanning tools may don't know what open source maintainers would like to receive from their tools. Maybe there is one tool that is specially used and specially wrong on this and that open source maintainers receive lots of false positives? We can analyze it and give specific feedbac. Maybe we could maintain documentation that explains how to manually verify some findings usually reported by tools, and we can ask tool developers to link to it, just to help the people that use that tool (I'm talking specially about any kind of scanning tool) to verify manually findings. The idea would be identify, prioritize, choose push initiatives that would improve tooling, maybe with documentation and code, in a way that open source maintainers receive less false positives from those tools, and for the vulnerabilities they receive, better information.

[MarcinHoppe]

This sounds like an initiative that could span several different OpenSSF WGs. Perhaps a better place to have this conversation would the OpenSSF Technical Advisory Council?

[MarcinHoppe]

That said, I think one aspect we definitely want to address at some point are security policies and guiding security researchers how to provide a high quality report is definitely something we want to work on.

[fcanogab]

Ok. Having that into account, I think it is better to just close this issue by the moment and maybe work on it from within other initiatives. Thanks!