Proof-of-concept for using oss-fuzz as a part of testing

[adaFPF]

OSS-fuzz looks like a lightweight avenue for adding some fuzzing into our automated testing. We're going to focus on Redwood as a source of test cases.

We will be creating use-case-specific build configs that will be committed to the linked repo upon project acceptance. Resulting build artifacts will be created by and tested in an OSS-fuzz-managed instance of ClusterFuzz, and findings will be reported there confidentially for up to 90 days, or upon earlier confirmation of closure on our end.

I'll be looking for time with a SecureDrop developer to:

• Help me better understand our current build system
• Provide guidance on selecting specific functions that are good candidates for this proof-of-concept

Relevant project docs:

• The architecture doc gives a good overview of the process itself: https://google.github.io/oss-fuzz/architecture/
• This page gives guidance for how to integrate as smoothly / effectively as possible: https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/
• Detailed integration guidance, including language-specific sections: https://google.github.io/oss-fuzz/getting-started/new-project-guide/

[legoktm]

Hi @adaFPF! We definitely want to add fuzzing to our security tooling, would it make sense to start with local fuzzing first before working on OSS-Fuzz?

We're going to focus on Redwood as a source of test cases.

I think redwood is probably the easiest place to hook into things just because it's relatively self-contained, but as far as value, I'm not sure how much we'll gain. redwood is a small bridge (~550 LOC) between PyO3 and Sequoia. Sequoia already has their own fuzzing setup (see https://gitlab.com/sequoia-pgp/sequoia/-/tree/main/openpgp/fuzz?ref_type=heads) so I'm skeptical on how much we'd find or value we'd get (would love to be wrong on this!)

Other potential fuzzing targets:

• messages being viewed in SD Client
• journalist replies being viewed on in the Source Interface, e.g. can XSS happen? (less important because we trust journalists)

Tangentially related, there's also stuff like https://github.com/freedomofpress/securedrop-workstation/issues/614 pending - I don't think we'd be fuzzing webp libraries ourself but might be part of our overall solution.

[adaFPF]

re: why oss-fuzz

I know very little about how SD does testing, and figuring that out adds time and complexity for me. My thoughts were that OSS-Fuzz would let us get some automated fuzzing while dodging that burden by not requiring changes to our test infra or processes. I figured it struck a good balance between how immediately useful it might be and how much time I'd need with someone explaining stuff to me to make it work.

If y'all are hesitant on OSS-Fuzz, I'd like to leave automation a separate concern. Besides the aforementioned knowledge lift I'd need, folks keep talking about moving our CI to GHA and if we do want to automate in our own pipeline the lazy mathematician in me would like to minimize the chance we'd have to implement it twice.

Either way, if folks are up for rescoping such that PoC means "proving this is doable and valuable with manual testing," I'm down. If that's the preferred path, any objections to using Atheris? If we do end up wanting to integrate with OSS-Fuzz, that's what they use for testing against python code and would thus make that path smoother.

re: good test candidates

Redwood was @zenmonkeykstop 's suggestion when I chatted with him about it. I think the self-contained aspect was part of why he thought it'd make a good use case for a proof-of-concept regardless of whether it would be likely to surface bugs.

I'm not wedded to it if you prefer some other options. I'd like our test targets to ideally be:

• Low complexity to keep finding interpretation as straightforward as possible
• Stuff that doesn't require me to build or install SD myself

In other words, if we're de-scoping to manual testing, picking a handful of candidate functions that I can write some harnesses for and try them out within a local dev env would be ideal.

[adaFPF]

Reading between the lines of your suggested candidates, I'm guessing you're interested in revisiting #6617. I'm not strictly opposed to it, but given what I already articulated, I'm betting it'd be a pretty heavy lift for now-me on my own. ZAP does lots of stuff besides input fuzzing, so moving forward here wouldn't preclude us from getting value out of it down the line.

[zenmonkeykstop]

IMO the originally-suggested approach is still sound. Redwood is pretty small LOC-wise but that's what makes it a good option for testing the waters. I'm also thinking in terms of future development - it's very likely that we'll be using Rust for any next-gen SD development, so having experience with fuzzing and oss-fuzz will be of benefit there.

I'm also not convinced that we won't get much benefit from it bug-hunt-wise. We (and other folks) got burned by a recent GPG key handling issue in rpm, which is also now wrapping Sequioa iirc. They're out there.

[legoktm]

I had the chance to catch up with @adaFPF today - starting with redwood sounds like a good plan to me.

[nwalfield]

IMO the originally-suggested approach is still sound. Redwood is pretty small LOC-wise but that's what makes it a good option for testing the waters. I'm also thinking in terms of future development - it's very likely that we'll be using Rust for any next-gen SD development, so having experience with fuzzing and oss-fuzz will be of benefit there.

I'm also not convinced that we won't get much benefit from it bug-hunt-wise. We (and other folks) got burned by a recent GPG key handling issue in rpm, which is also now wrapping Sequioa iirc. They're out there.

Hi @zenmonkeykstop , I'm the author of sequoia and rpm-sequoia. Could you tell me what issue you are talking about. Thanks!

[zenmonkeykstop]

Oh, sorry @nwalfield, I missed the comment - I was referring to https://github.com/rpm-software-management/rpm/issues/2577, which I believe was recently fixed.