search

rust-fuzz / cargo-fuzz

Command line helpers for fuzzing
https://rust-fuzz.github.io/book/cargo-fuzz.html
Apache License 2.0
1.51k stars 107 forks source link

Add support for AFL #1

Open Manishearth opened 7 years ago

Manishearth commented 7 years ago

Would be nice if we could cargo fuzz --fuzzer afl on binary crates. Or something. Might need to set up the whole LLVM shebang.

cc @frewsxcv

frewsxcv commented 7 years ago

~This would be pretty easy to do if we figured out https://github.com/rust-fuzz/libfuzzer-sys/issues/5~

frewsxcv commented 7 years ago

I think the best strategy here is to make a change to rustc to add in the afl LLVM pass behind a rustc debug flag.

Relevant IRC rust-internals discussion: https://botbot.me/mozilla/rust-internals/2017-02-26/?msg=81593902&page=3

https://github.com/rust-lang/rust/tree/master/src/rustllvm

https://github.com/rust-lang/rust/tree/master/src/librustc_llvm

whitequark commented 7 years ago

What's the advantage of afl over libfuzzer?

nagisa commented 7 years ago

@whitequark the only one I know is that it, being a out-of-process fuzzer, can trivially run even in presence of crashes and avoids giving the crashing inputs it has already seen.

libfuzzer struggles with that use-case, sadly.

frewsxcv commented 7 years ago

What's the advantage of afl over libfuzzer?

In my opinion, the UI is a lot more informative and easier to understand: http://lcamtuf.coredump.cx/afl/status_screen.txt . Also from what I've seen, there's a larger ecosystem of tools built around AFL.

frewsxcv commented 6 years ago

okay, incorporating afl into cargo-fuzz is now doable.

yesterday, i opened this issue in rust-lang/rust. i was informed by @alex in that thread that it's now possible to use afl without relying on afl's own llvm pass and instead use llvm's trace-pc-guard feature. i wasn't aware of this, tried it out this morning, and got it working!

in particular, here's how to get it setup:

  1. compile afl with the trace-pc feature
    1. download afl
    2. AFL_TRACE_PC=1 make clean all as per this
  2. compile the afl runtime
    1. (from the afl source directory)
    2. gcc -c -O1 -fPIC -fno-omit-frame-pointer llvm_mode/afl-llvm-rt.o.c -fpermissive (copied some of these flags from oss-fuzz)
    3. ar r libafl-llvm-rt.a afl-llvm-rt.o.o
  3. setup/compile a fuzz target
    1. just to demonstrate that this all works, just cargo new --bin foo printing 'hello world' is sufficient to show that afl can even run, albeit not very interesting
    2. export RUSTFLAGS="-Cllvm-args=-sanitizer-coverage-level=3 -Cllvm-args=-sanitizer-coverage-trace-pc-guard -Cpasses=sancov"
    3. cargo rustc -- -l afl-llvm-rt -L <directory where libafl-llvm-rt.a is>
  4. run afl 🎉
    1. <afl source directory>/afl-fuzz -i <corpus dir> -o out target/debug/<crate name>

the main work left here is the integration into cargo-fuzz

some questions:

some thoughts:

frewsxcv commented 6 years ago

Published afl.rs 0.2 yesterday: https://users.rust-lang.org/t/announcing-afl-rs-0-2-bindings-for-american-fuzzy-lop/13981 which should make it easier to complete this issue now

Manishearth commented 6 years ago

in a world where cargo-fuzz can handle multiple fuzzers, what is the directory structure in fuzz/? for example, where should the output of afl go in the fuzz directory?

We have an artifacts/ and corpus/ folder that turn up there. We can make the artifacts/ folder into artifacts/libfuzzer and artifacts/afl if we wish.

unlike libfuzzer, afl-fuzz relies on bytes coming in through stdin, so we'll need to update our fuzz_target macro to account for that

Can we make the macro paper over this to provide the same API?

Manishearth commented 6 years ago

Alternatively we can make AFL work on cargo examples directly.