search

google / oss-fuzz

OSS-Fuzz - continuous fuzzing for open source software.
https://google.github.io/oss-fuzz
Apache License 2.0
10.15k stars 2.16k forks source link

boost: add fuzzers for beast library #11994

Closed tyler92 closed 3 weeks ago

tyler92 commented 1 month ago

boost_beast_request_fuzzer for fuzzing HTTP requests parser boost_beast_response_fuzzer for fuzzing HTTP responses parser boost_beast_ws_server_fuzzer for fuzzing WebSocket server

github-actions[bot] commented 1 month ago

tyler92 is a new contributor to projects/boost. The PR must be approved by known contributors before it can be merged. The past contributors are: maflcko, TheZ3ro, DonggeLiu, Navidem, inferno-chromium, Dor1s, bshastry (unverified), pauldreik (unverified), kcc, nevir (unverified)

tyler92 commented 1 month ago

Ideally, I would add a corpus for the fuzzer. Do you recommend adding a zip archive here or there is a chance that the fuzzer will be executed with an existing public corpus?

tyler92 commented 1 month ago

I've added two zip archives. They are quite tiny, but please let me know if it's not a recommended way

DaveLak commented 1 month ago

@tyler92 I'm not a maintainer here but I can offer some guidance.

I've added two zip archives. They are quite tiny, but please let me know if it's not a recommended way

The OSS-Fuzz project typically avoids and discourages adding corpora and dictionary files to this repo because it bloats the size of the Git repo, making it slower and more resource (e.g., disk space) intensive to download/clone for everyone (including the ClusterFuzz bots.)

In fact, there seems to be a CI check for disallowed zip files that is broken and has a fix proposed in: https://github.com/google/oss-fuzz/pull/12008. If working it would have failed on your latest commit with the message "Don't commit seed corpora into the ClusterFuzz repo,they bloat it forever."

Do you recommend adding a zip archive here or there is a chance that the fuzzer will be executed with an existing public corpus?

The best approach is probably to ask if the upstream maintainers would be interested in setting up somewhere to host corpora outside of this repo (and in fact, I bet OSS-Fuzz maintainers would appreciate if upstream also took on maintenance of the fuzz targets.)

I like how Bitcoin Core does it in: https://github.com/bitcoin-core/qa-assets

In fact, I set up something similar for GitPython: https://github.com/gitpython-developers/qa-assets

Hope that helps!

DaveLak commented 1 month ago

@tyler92 sorry for the double pings. I forgot to answer this part:

is a chance that the fuzzer will be executed with an existing public corpus?

ClusterFuzz will generate a corpus for each target during normal runs, even if no seed corpus is provided. It can take a few days and start empty (i.e., won't use a public corpus as a seed) initially, but after a few successful runs corpora will be generated and persisted for future use.

tyler92 commented 1 month ago

@tyler92 sorry for the double pings. I forgot to answer this part:

is a chance that the fuzzer will be executed with an existing public corpus?

ClusterFuzz will generate a corpus for each target during normal runs, even if no seed corpus is provided. It can take a few days and start empty (i.e., won't use a public corpus as a seed) initially, but after a few successful runs corpora will be generated and persisted for future use.

This is good news, thank you for the information! I've removed zip files, now there is only one commit with two targets. As far as I could see all boost targets are there and not in the boostorg repository. I agree with you that it's better to move targets from oss-fuzz repo. I believe we can proceed with the current approach and move targets from here in a separate MR if the maintainers don't mind.

tyler92 commented 1 month ago

There was the same issue before with https://github.com/google/oss-fuzz/pull/11477 :

The fuzzers are temporary committed in oss-fuzz, but we are in touch with upstream maintainers to add the harnesses directly in their repositories. Sadly the Boost ecosystem is heavily fragmented so this process will take some time.

tyler92 commented 3 weeks ago

I think the MR is stuck. Who can I contact for feedback?

tyler92 commented 3 weeks ago

The best approach is probably to ask if the upstream maintainers would be interested in setting up somewhere to host corpora outside of this repo (and in fact, I bet OSS-Fuzz maintainers would appreciate if upstream also took on maintenance of the fuzz targets.)

Thanks for the advice, it's ready: https://github.com/google/oss-fuzz/pull/12109. I will close this PR because we have a better alternative.