search

jaraco / zipp

MIT License
61 stars 53 forks source link

Continuous fuzzing by way of OSS-Fuzz #83

Closed DavidKorczynski closed 1 year ago

DavidKorczynski commented 2 years ago

Hi,

I was wondering if you would like to integrate continuous fuzzing by way of OSS-Fuzz? Fuzzing is a way to automate test-case generation and can be used to find unexpected exceptions in Python. In this PR https://github.com/google/oss-fuzz/pull/8267 I did an initial integration into OSS-Fuzz and the current fuzzer targets zipp.Path using a zipfile generate from fuzzer data. The fuzzing engine used by OSS-Fuzz is Atheris.

If you would like to integrate, the only thing I need is a list of email(s) that will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats. Notice the emails affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

jaraco commented 2 years ago

Hi David. Thanks for offering to fuzz this project. I'd welcome the effort, though I don't think email notifications would be the best. Is there a reason you couldn't instead register any failures as issues with this project? If email is the best/only means of notification, we might consider setting up a list (google group or similar) where notifications could be registered and then someone (maintainers or contributors) could periodically review that list and convert the reports to issues.

jaraco commented 1 year ago

On further consideration, I don't think email notifications to a human is the best way to integrate a test suite. Instead, either someone should monitor the runs that come out of the fuzzer as it runs in google/oss-fuzz or those tests should be ported to this project's test suite. I currently manage over 150 OSS Python projects, and doing that in a scalable way has required dropping even the email notifications for Github. Instead, I check in periodically on the various projects. Unless the fuzz results can integrate into the project's test suite, it won't integrate well into the maintenance workflows. I welcome you or someone else to monitor/subscribe to the fuzzer runs and report issues here if they emerge or to propose an integration into this project's CI runs.

Thanks again.