search

jarro2783 / cxxopts

Lightweight C++ command line option parser
MIT License
4.25k stars 590 forks source link

Integrate with google/oss-fuzz #387

Closed nathaniel-brough closed 1 year ago

nathaniel-brough commented 1 year ago

Hey I'd like to suggest adding cxxopts to google/oss-fuzz. If you aren't familiar with fuzz testing, here is a bit of a run down (from Wikipedia);

In programming and software development, fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks.

Google offers a free continuous fuzzing service called OSS-fuzz. If cxxopts is integrated into oss-fuzz, the fuzz tests under cxxopts will be built and then run once a day, to search for bugs and vulnerabilities in cxxopts. This service can be integrated with the CI for cxxopts, so that the fuzz tests are run for 10min or so for every pull request, preventing buggy code from being merged.

I've opened up a pull request to add a basic fuzz-testing harness here https://github.com/jarro2783/cxxopts/pull/386. If you are keen on adding cxxopts to oss-fuzz I'd be happy to champion the integration :)

jarro2783 commented 1 year ago

That looks interesting. I'll take a look at your PR.

nathaniel-brough commented 1 year ago

Hey @jarro2783, I've had some time to start integrating cxxopts with OSS-fuzz. I've got a draft PR here https://github.com/google/oss-fuzz/pull/9679, all I need from you is an email address associated with a google account to add to the primary_contact. Does the @gmail.com address in the git logs work for you? You can read about why a google account is required here.

As an additional step, if you are happy to go ahead with integrating with OSS-fuzz I'll need you to comment on the PR mentioning that you approve of the request to join the program :)

jarro2783 commented 1 year ago

Yes you can use the email address that I use in my commits.

nathaniel-brough commented 1 year ago

Yes you can use the email address that I use in my commits.

Done

nathaniel-brough commented 1 year ago

So this is now integrated into oss-fuzz. I've just opened up a PR #392 to run the fuzzer for 10min or so to catch shallow bugs in pull requests. This is in addition to the once per day fuzzing in oss-fuzz.

On a different note, are you receiving notifications from OSS-fuzz about failing coverage collection? I won't receive these notifications I'm not listed as the primary_contact. But I checked and it looks like there might be some issue with the code-coverage collection. I might give it a couple more days, I think code coverage is calculated every other day, so it might just be a timing thing.

jarro2783 commented 1 year ago

Looks like this issue is all finished with. Thanks for the integration.