Integrating with OSS-Fuzz

[Google-Autofuzz]

Greetings Halide developers and contributors,

We’re reaching out because your project is an important part of the open source ecosystem, and we’d like to invite you to integrate with our fuzzing service, OSS-Fuzz. OSS-Fuzz is a free fuzzing infrastructure you can use to identify security vulnerabilities and stability bugs in your project, like #4047 and #4003. OSS-Fuzz will:

• Continuously run all the fuzzers you write.
• Alert you when it finds issues.
• Automatically close issues after they’ve been fixed by a commit.

Many widely used open source projects like OpenSSL, FFmpeg, LibreOffice, and ImageMagick are fuzzing via OSS-Fuzz, which helps them find and remediate critical issues.

Even though typical integrations can be done in < 100 LoC, we have a reward program in place which aims to recognize folks who are not just contributing to open source, but are also working hard to make it more secure.

We want to stress that anyone who meets the eligibility criteria and integrates a project with OSS-Fuzz is eligible for a reward.

If you're not interested in integrating with OSS-Fuzz, it would be helpful for us to understand why—lack of interest, lack of time, or something else—so we can better support projects like yours in the future.

If we’ve missed your question in our FAQ, feel free to reply or reach out to us at oss-fuzz-outreach@googlegroups.com.

Thanks!

Julien, OSS-Fuzz Team

[alexreinking]

We have some fuzz tests already. I'd be happy to try running them on your service.

[alexreinking]

A major issue I'm finding with integrating OSS-Fuzz into our project is that we don't really ingest strings anywhere. Our existing fuzz tests always build well-typed expressions and the error checking in our IR data structures aborts the process if constructing an ill-typed IR node is attempted.

Presumably, ill typed expressions and bad parser inputs will be the common case with libFuzzer/AFL style fuzzers.

We could ignore the inputs that the fuzzers give us and just generate a batch of other inputs. Really the paradigmatic issue is that our current fuzz testing is more property testing a la QuickCheck than (guided) random inputs like AFL.

Any advice would be appreciated.

[steven-johnson]

We could ignore the inputs that the fuzzers give us and just generate a batch of other inputs

Or use the inputs to form a seed for our "fuzzer" tests, to improve repeatability. But yes, I agree, I'm not sure if our code is as obviously a good fit as some others.

[abadams]

fuzz_simplify uses an rng to build the expression. It could instead consume a stream of tokens to make its choices. Then there would be coherence of a sort that afl might be able to exploit.

We'd only get coverage of the simplifier though that way, but apps/random_pipeline could consume a stream of tokens in the same way, which would give us coherent coverage of the entire compiler

[alexreinking]

It could instead consume a stream of tokens to make its choices. Then there would be coherence of a sort that afl might be able to exploit.

This is a good idea, and one worth trying. Maybe just let each token be a single character? Then each character acts as an instruction to a sort of abstract machine that spits out Halide syntax.

[steven-johnson]

Has anything ever come of this?

[silvergasp]

fuzz_simplify uses an rng to build the expression. It could instead consume a stream of tokens to make its choices. Then there would be coherence of a sort that afl might be able to exploit.

We'd only get coverage of the simplifier though that way, but apps/random_pipeline could consume a stream of tokens in the same way, which would give us coherent coverage of the entire compiler

Just as a bit of an update, I'm working on getting halide supported in OSS-fuzz, I've got an initial PR open in #7512, and I'll open a draft PR on OSS-fuzz shortly to integrate with the fuzzing service.

[silvergasp]

Now that #7512 has merged I'm going to tackle the other RNG based fuzz harnesses converting them over so that they are libfuzzer compatible.

• [x] ./test/correctness/fuzz_cse.cpp #7543
• [ ] ./test/correctness/fuzz_bounds.cpp #7549
• ./test/correctness/fuzz_float_stores.cpp (Abandoned See comments below)

[silvergasp]

@steven-johnson I started trying to port over ./test/correctness/fuzz_float_stores.cpp to libfuzzer but before I started making any modifications I ran the test and got this error.

JIT session error: Unsupported x86-64 relocation type 22: R_X86_64_GOTTPOFF
Internal Error at /home/nathaniel/projects/gh.com/silvergasp/Halide/src/JITModule.cpp:226 triggered by user code at : Condition failed: addr: Failed to materialize symbols: { (main, { __orc_deinit_func.f0, __orc_init_func.f0, sancov.module_ctor_8bit_counters, asan.module_dtor, $.f0.__inits.0, f0_metadata, f0_argv, f0, asan.module_ctor }) }

Things that I tried so far include;

• Linking with --ld-path=/usr/bin/ld.lld-15 I thought it might be something to do with the linker.
• Linking against llvm15 and llvm16

Honestly I'm kind of stumbling around in the dark on this one, I know next to nothing about llvm+jit compilation in general so this is very much outside of my area of expertise. Do you have an thoughts/intuition on where this issue would be coming from? To be honest I'm not even sure that this fuzzer would benefit much from being ported over to libfuzzer. The only thing that I can see that is really directly fuzzable would be changing the size of the test.

I might be able to do more fuzzing with some more intrusive changes into src/FuzzFloatStores.cpp but at the moment I only have a vague idea about how that might work.

[abadams]

This one isn't a fuzzer in the same sense as the others so I don't think it makes sense to port it over. This test exercises a feature that masks off the low bit of all floating point stores as a way to flush out tests that are too sensitive to floating point precision issues. It's sort of the opposite of strict_float.