Proposal: Integrate with google/oss-fuzz for continuous fuzz testing?

silvergasp commented 6 months ago

Hey yoga team,

I've recently become interested in yoga. I'd like to suggest and champion an effort to set up some basic fuzz-testing and combine it with google/oss-fuzz for continuous fuzzing. I'm fully aware that you are very busy people and I don't want to overload your review/maintenance capacity. Is this a bad time to discuss potential security/reliability improvements?

If you're not familiar with fuzzing or oss-fuzz I've included a few brief notes below.

Benefits of Fuzz-Testing

Dynamic Code Testing: Fuzz-testing challenges systems with unexpected data, aiming to identify vulnerabilities or bugs. It’s akin to an exhaustive stress-test for the code.
Detecting Hidden Vulnerabilities: It can uncover potential weaknesses that may not be evident in routine tests.
Continuous and Automated Testing: With tools like Google’s OSS-Fuzz, fuzz-testing can be automated, running continuously on distributed systems, ensuring daily resilience checks.

Google/oss-fuzz for Continuous Fuzzing

Automated Fuzzing: OSS-Fuzz undertakes comprehensive fuzz-testing daily on a distributed cluster.
Detailed Reporting: OSS-Fuzz offers exhaustive reports in case of detected anomalies, enabling effective action.

I’d be more than happy to lead the effort in integrating fuzz testing with the yoga and assist in any way required.

Prior integrations

There have been a number of previous integrations completed with facebook repositories and google/oss-fuzz including;

facebook/time
facebook/zstd
facebookexperimental/starlark-rust (this was me)
facebook/proxygen
facebook/hermes
facebook/rocksdb

As a proof of concept I created a couple of super simple fuzz harnesses in #1537.

NOTE: Adding fuzz-testing and integrating with google/oss-fuzz was previously suggested here https://github.com/facebook/yoga/pull/1055 and was rejected. I think I've addressed the concerns raised in the first PR. While the original PR contained what was probably a higher performance fuzzer, the new fuzzer should be easier to integrate and doesn't introduce multiple sources of truth.

NickGerleman commented 6 months ago

Thank you for raising this, and contributing a PR implementing it. I think it is a good idea, so long as we make sure it isn't disrupted by any unrelated Yoga changes.

silvergasp commented 5 months ago

As a follow up I'm going to start the integration with google/oss-fuzz if you are still keen on it. There is an application/integration process which I can complete on your behalf. To do this I'll need a couple of things from you;

A google/gmail email address that I can use for the primary contact. As far as I'm aware meta/facebook emails meet this requirement.
You to comment on the PR that I'll create that will form the integration/application saying that you approve of the integration.

Given the popularity of yoga I'm reasonably confident that it'll be accepted into oss-fuzz, but there is a non-zero chance that it could be rejected.

silvergasp commented 5 months ago

So I've got a draft PR ready to go integrating with OSS-fuzz in https://github.com/google/oss-fuzz/pull/11533. The only thing I need to do now is add a primary contact. I've added oss-fuzz@fb.com which seems to be CC'd on most of the other facebook integrations.

NickGerleman commented 4 months ago

Thanks @silvergasp. Is there anything left to action on from our side?

silvergasp commented 4 months ago

Yeah, all I need is the email address that you'd like to use to get access to the bug tracker and oss fuzz dashboard. Just as a note, it'll be stored as plain text on the oss fuzz git repo. After that everything should be good to go.

NickGerleman commented 4 months ago

Reusing oss-fuzz probably makes sense if it is already being used, though I’m not actually sure where it is funelling to. I can try to find that out.

silvergasp commented 4 months ago

Easy do you want me to add your email to the config? That will give you personal access to the dashboard/bug tracker?

NickGerleman commented 4 months ago

I appreciate it. Could you use this address? https://github.com/facebook/yoga/commit/cc2a87d4f95d2deac3f4ffed02ca4c130aec70ba.patch

silvergasp commented 1 month ago

Ah sorry I missed this notification. I've pulled the trigger on the PR over at oss-fuzz now. All that is needed is to wait for the oss-fuzz team to approve.

facebook / yoga