search

google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.11k stars 269 forks source link

Extending benchmarks to Rust targets #1874

Open addisoncrump opened 1 year ago

addisoncrump commented 1 year ago

As discussed in FUZZING'23, we want to extend Fuzzbench to allow for Rust targets. Not all fuzzers will be compatible, but we would like to define a standard interface for those which are. We suggest the following and ask for input so that future fuzzers may also support Rust benchmarks.

Rust harnesses

Most Rust fuzz harnesses built with cargo-fuzz should be straightforward to modify such that they emit staticlib build units instead of executable binaries. In recent versions of libfuzzer-sys, the common crate for providing libfuzzer bindings to cargo-fuzz targets, you can choose to not link libfuzzer and instead only use the function names. This turns a Rust harness into a classical LLVMFuzzerTestOneInput-style harness. We propose to build Rust benchmarks in this way and link against the static libraries like existing benchmarks.

Build flags and compilers

Rust fuzzers may use a specialised rustc by exporting the CARGO_BUILD_RUSTC environmental variable. In addition, fuzzers using default rustc may export specific build flags with the RUSTFLAGS environmental variable. The latter is done by cargo-fuzz.

Detecting Rust compatibility

If a fuzzer does not set the RUSTFLAGS or CARGO_BUILD_RUSTC variable before performing the build, we should assume that it does not support Rust (no instrumentation passes would be executed). Alternatively, we should export a fuzzer.compatible_languages() function in each fuzzer, which may future-proof in the case of e.g. JavaScript/Java-specific fuzzers.

jonathanmetzman commented 1 year ago

Hmm...I guess we could also use the blacklisting feature to disable incompatible fuzzers. and it would keep the API thinner. Let me think about this.

jonathanmetzman commented 1 year ago

I like the idea of having benchmarks opt-in to be part of a "group" (e.g. rust) and then fuzzers can say which groups they want to be fuzz.

addisoncrump commented 1 year ago

I think it makes the most sense if you ever want to extend to e.g. JS/Python/firmware etc. fuzzing.

jonathanmetzman commented 1 year ago

Can we start with blacklisting fuzzers for now? I'm happy to accept rust targets, though I'm not sure there's real benefit (hard to believe we could write a fuzzer better/worse for rust than C/C++)