Support Honggfuzz - Githubissues

PaulGrandperrin commented 6 years ago

Honggfuzz looks to be a great competitor to libFuzzer and AFL.

Also, the instrumentalization procedure seems to be very similar to libFuzzer so it might be quite easy to add support ;-)

A copy-paste of the description:

A security oriented, feedback-driven, evolutionary, easy-to-use fuzzer with interesting analysis options. See USAGE for the description of command-line options.

It's multi-process and multi-threaded: no need to run multiple copies of your fuzzer, as honggfuzz can unlock potential of all your available CPU cores with one process. The file corpus is automatically shared and improved between the fuzzing threads.
It's blazingly fast when in the persistent fuzzing mode). A simple/empty LLVMFuzzerTestOneInput function can be tested with up to 1mo iterations per second on a relatively modern CPU (e.g. i7-6700K)
Has a solid track record of uncovered security bugs: the only (to the date) vulnerability in OpenSSL with the critical score mark was discovered by honggfuzz. See the Trophies paragraph for the summary of findings to the date
Uses low-level interfaces to monitor processes (e.g. ptrace under Linux). As opposed to other fuzzers, it will discover and report hijacked/ignored signals (intercepted and potentially hidden by signal handlers)
Easy-to-use, feed it a simple corpus directory (can even be empty) and it will work its way up expanding it utilizing feedback-based coverage metrics
Supports several (more than any other coverage-based feedback-driven fuzzer) hardware-based (CPU: branch/instruction counting, Intel BTS, Intel PT) and software-based feedback-driven fuzzing methods known from other fuzzers (libfuzzer, afl)
Works (at least) under GNU/Linux, FreeBSD, Mac OS X, Windows/CygWin and Android
Supports the persistent fuzzing mode (long-lived process calling a fuzzed API repeatedly) with libhfuzz/libhfuzz.a. More on that can be found here
Can fuzz remote/standalone long-lasting processes (e.g. network servers like Apache's httpd and ISC's bind), though the persistent fuzzing mode is suggested instead: as it's faster and multiple instances of a service can be fuzzed with this
It comes with the examples directory, consisting of real world fuzz setups for widely-used software (e.g. Apache and OpenSSL)

PaulGrandperrin commented 6 years ago

I just successfully made honggfuzz work with Rust code :-) I would be happy to implement myself honggfuzz support if you want.

frewsxcv commented 6 years ago

I would be happy to implement myself honggfuzz support if you want.

🤩🤩🤩 this would be amazing! how much work was it to get it working? i haven't tried out honggfuzz yet, but i've been meaning to! looks like a great fuzzing utility

PaulGrandperrin commented 6 years ago

I just got it working on a simple test very easily. I don't remember exactly the steps but it was something quick and dirty like that:

build honggfuzz as libhfuzz.a
use libfuzzer wrapper code to export LLVMFuzzerTestOneInput function
build with cargo rustc with the correct arguments

It does not, however, work with bigger projects. For example on a project, the dependancy backtrace-sys will fail to compile with sancov enabled because of missing symbols (it's trying to build a binary, but does not link it to libhfuzz.a). However, I guess it's unrelated to compiling with honggfuzz. I'm curious about how cargo-fuzz/libfuzzer solve this problem.

PaulGrandperrin commented 6 years ago

Hi, I've been working hard this week on building the honggfuzz crate. I'm pretty happy with what it looks like now, it's pretty minimal (does not handle the creation of boilerplate code), but I think the API is easy to understand and use.

https://crates.io/crates/honggfuzz https://github.com/PaulGrandperrin/honggfuzz-rs

I made a recording of how it works:

Don't hesitate to give me feedback :-)

frewsxcv commented 6 years ago

this is all very cool @PaulGrandperrin, nice work! so now we have both afl.rs and honggfuzz-rs which work great as standalone fuzzing tools, but do you think we should try to incorporate one or both into cargo-fuzz as well? i'd be very interested in seeing that happen with afl.rs, but i haven't had much time to think about how that'd work, or what changes would need to be made to cargo-fuzz

also, i don't want this to sound like i'm pressuring you to do so, but if you did have interest in moving the honggfuzz repo to the rust-fuzz github organization, let me know and we can make that happen

PaulGrandperrin commented 6 years ago

Hi @frewsxcv ! While working on porting Honggfuzz to Rust I first worked on the cargo-fuzz codebase but then I thought it would be better to first build an independent project instead. Here is my reasoning:

if cargo-fuzz becomes a generic fuzzing framework it will probably be limited by the lowest common denominator of features. So I think it would be nice to also have more specific projects tailored to the specificities of each fuzzer.
cargo-fuzz is, as I see it, an opinionated high-level framework, (kind of like ruby-on-rails). There is nothing wrong with that but some people prefer to have more control over how to integrate with their projects. honggfuzz-rs dictates a little bit less how to integrate with projects.
I, therefore, think the best solution would be to build one crate for each project (just like now) with a quite minimalist and non-opinionated library-like design and then abstract them with a very high-level crate (like cargo-fuzz) for the people who don't want to go into the specifics of each fuzzers.

Also, I want to gain some experience with honggfuzz and building crates in general before thinking too much about the next steps.

However, I want to confirm that it also was my objective from the beginning to build something that could eventually be integrated into something like cargo-fuzz. I would also be happy to move the project under the "rust-fuzz" umbrella.

frewsxcv commented 6 years ago

all sounds great, thanks for writing it out!

I would also be happy to move the project under the "rust-fuzz" umbrella.

again, no pressure, but if you did want to do this, you should be an admin for the rust-fuzz org and should be able to migrate the repository over

PaulGrandperrin commented 6 years ago

Awesome! Done. Should we close this issue now or when honggfuzz is available from cargo-fuzz itself?

killercup commented 6 years ago

Given that @PaulGrandperrin did such an awesome job of generalizing the rust-fuzz/targets to three fuzzers in https://github.com/rust-fuzz/targets/pull/102 I'm wondering what it would take to do the same for cargo-fuzz.

Is it possible to have users put all their fuzz code into a fuzz_target! macro call, and then have cargo-fuzz generate fuzzer-specific code? We already require nightly for some fuzzers IIRC, so we could totally use proc macros if we wanted to.

PaulGrandperrin commented 6 years ago

I think it will be difficult to really have something API compatible in the short term, even using proc macros. I started an RFC to start to lay down a path toward that goal but I think it will take time to get there. For example, libfuzzer requires #[no_main] but I don't think you can generate that from a macro because of macro hygiene (I might be wrong).

However, in the mean time, cargo-fuzz could instantiate templates just like in https://github.com/rust-fuzz/targets, and then compile thoses.

0xalpharush commented 1 month ago

What is the status of https://github.com/rust-fuzz/cargo-fuzz/issues/1 and this issue? I see the RFC but wondering if there has been further work or discussion

rust-fuzz / cargo-fuzz

Support Honggfuzz #140