intel / cve-bin-tool

The CVE Binary Tool helps you determine if your system includes known vulnerabilities. You can scan binaries for over 200 common, vulnerable components (openssl, libpng, libxml2, expat and others), or if you know the components used, you can get a list of known vulnerabilities associated with an SBOM or a list of components and versions.
https://cve-bin-tool.readthedocs.io/en/latest/
GNU General Public License v3.0
1.2k stars 457 forks source link

test: basic execution test for EPSS #4484

Open terriko opened 1 week ago

terriko commented 1 week ago

EPSS tests are surprisingly hard because the scores change constantly, and we had to disable some of the tests originally written because they were too unstable. But as a result, the bug in #4473 slipped through because we didn't have a check for the most basic of errors.

What I'd like is a test that runs a scan with epss enabled, then checks the logs to make sure no errors appear in the log, I believe this should be pretty easy, since you can use caplog.set_level(logging.ERROR) and check that nothing occurs in there. You should be able to look at the existing test in test__source_epss.py to give you a hint on how to call just the epss related code, or you could look at test_cli.py for how to do a full command line run. Remember to strip down what you test as much as possible to make the test run faster: ideally we want to look up as little as possible because this test is mostly about making sure there's no obvious errors we're missing.

This may require some reading to make sure you understand how to use pytest and how to call the correct code, but I think it should be beginner-friendly so I'm tagging it as a "good first issue" and will put the new contributor tips below. I'm also tagging it for hacktoberfest. If you're interested in doing this as part of the contest, make sure you do it within their timeframe and follow their rules. (Most notably: don't create the PR before October 1 or it won't count)

Short tips for new contributors:

Claiming issues:

weichslgartner commented 1 week ago

I would be happy to work on this issue.