fix: improve OSV config for OSSF Scorecard

[terriko]

The scorecard is giving us a lower score because it claims we have OSV vulnerabililities:

A sampling:

Warn: Project is vulnerable to: RUSTSEC-2021-0139
Warn: Project is vulnerable to: RUSTSEC-2021-0145
Warn: Project is vulnerable to: GHSA-f85w-wvc7-crwc / RUSTSEC-2022-0078
Warn: Project is vulnerable to: RUSTSEC-2020-0159
Warn: Project is vulnerable to: GHSA-jqwc-c49r-4w2x
Warn: Project is vulnerable to: GHSA-7f6x-jwh5-m9r4
Warn: Project is vulnerable to: RUSTSEC-2020-0168
Warn: Project is vulnerable to: GHSA-3gxf-9r58-2ghg / RUSTSEC-2023-0022
Warn: Project is vulnerable to: GHSA-6hcf-g6gr-hhcr / RUSTSEC-2023-0024
Warn: Project is vulnerable to: GHSA-9qwg-crg9-m2vc / RUSTSEC-2023-0023
Warn: Project is vulnerable to: GHSA-xcf7-rvmh-g6q4

We don't actually use rust, for example, so these are almost certainly all coming from test/language_data which should not be scanned. There's some notes suggesting that we could add a config to improve these results and I'm hoping that this means that (unlike with dependabot) we could tell it not to scan that directory. If anyone's got time to look up how to do that config, that would be great.

[terriko]

If you believe the vulnerability does not affect your project, the vulnerability can be ignored. To ignore, create an osv-scanner.toml file next to the dependency manifest (e.g. package-lock.json) and specify the ID to ignore and reason. Details on the structure of osv-scanner.toml can be found on OSV-Scanner repository.

[chillerno1]

@terriko I think this is one I could pick up. Is there anything in particular I need to know (aside from your notes above) before diving into it?

[terriko]

@chillerno1 I think I brain dumped most of it into this issue, but here's a list of directories/files of interest:

We should never be scanning:

• The problem files are in test/language_data and that directory can be excluded.
• We could exclude all of test/ safely too, but I think all the false positives are from test/language_data at this time.

We should be scanning:

• requirements.txt
• dev-requirements.txt
• doc/requirements.txt
• and probably our javascript dependences (in cve_bin_tool/output_engine/js if OSV can. (they may not be in a format OSV supports)

Note that we are scanning that second group ourselves, so if for any reason we can't use the OSV tool on it that's not such a big deal since the scanning is still happening. If we can do both it's useful for us to spot differences in the tools and low-key validate that our own OSV import is working correctly (which, at this very moment, we know it has a bug that's being actively worked on).

[chillerno1]

@terriko Thanks, you're welcome to assign this to me. I hope you don't mind me asking a few questions as I work through it.

I've confirmed that adding test\language_data\osv-scanner.toml with [[IgnoredVulns]] tags, filters out the results.

[[IgnoredVulns]]
id = "RUSTSEC-2021-0139"
reason = "Vulnerability should be ignored as the project does not use Rust, file flagged is from test/language_data."

osv-scanner -r .\cve-bin-tool\
Scanning dir .\cve-bin-tool\
...
Loaded filter from: \cve-bin-tool\test\language_data\osv-scanner.toml
RUSTSEC-2021-0139 has been filtered out because: Vulnerability should be ignored as the project does not use Rust, file flagged is from test/language_data.

However ...

.. I'm hoping that this means that (unlike with dependabot) we could tell it not to scan that directory. If anyone's got time to look up how to do that config, that would be great.

From what I've read, there doesn't seem to be a way to exclude the entire directory from scanning. The configuration for ignoring seems to be quite limited. Each vulnerability needs to be tagged with [[IgnoredVulns]] in a toml file, where the dependency manifests are.

Thinking of a potential solution or workaround, maybe I could write a script/action that runs osv-scanner and populates osv-scanner.toml files for any vulnerabilities picked up in the tests/* dir? Alternatively, I'm happy to update them manually for now.

[terriko]

I'm already updating a lot of vulns manually thanks to dependabot, so I'd love a more long-term solution. We do need to be careful that in ignoring stuff in test/ we don't accidentally ignore something that also occurs in our non-test directories, but probably not too careful as they're being scanned with different tools already.