rust-fuzz / libfuzzer

Rust bindings and utilities for LLVM’s libFuzzer
Apache License 2.0
208 stars 44 forks source link

Support for arbitrary data types #26

Closed nagisa closed 7 years ago

nagisa commented 7 years ago

Not efficient[1], but that mostly needs work on the arbitrary crate, as opposed to work on libfuzzer-sys.

[1]: fuzzing struggles with e.g.

fuzz_target!(|data: String| { if data == "banana" { panic!() } });

and takes quite a while to find an input that fails the check (compared to fuzzing against raw bytes). It seems to me that to take full advantage of fuzzing structured data, one would have to adjust libFuzzer somehow or, perhaps, rewrite libFuzzer in rust having precisely structured fuzzing in mind.

nagisa commented 7 years ago

cc @Manishearth @frewsxcv for a review.

Manishearth commented 7 years ago

Why not just use the quickcheck stuff here? Bonus points if we can make a macro that also expands to a quickcheck test depending on some flag.

nagisa commented 7 years ago

Primary motivation is me not really wanting to deal with having to land stuff into quickcheck at the moment, because I need fast turn-around times for upcoming month.

Besides, quickcheck’s implementations of arbitrary are oriented towards quickcheck and random data, and those don’t necessarily work well for generating structured data from, say, a ring buffer as is done here.

In the end, I hope to migrate quickcheck itself to use the arbitrary crate, but it might end up being entirely unfeasible due to significantly differing requirements/desires/constraints.

Manishearth commented 7 years ago

@burntsushi thoughts?

fine with merging as is but would like to avoid duplication

BurntSushi commented 7 years ago

@Manishearth I don't have any specific thoughts, but I am strongly in favor of folks experimenting with improving quickcheck-style testing. Not everything in this realm fits cleanly with how the quickcheck crate is setup, and I suspect there are nice ergonomic wins to be had if the idea was given a refresh. (I might perhaps point folks towards auto-shrinking and the Hypothesis work.)

Manishearth commented 7 years ago

Cool. I'd love for the two crates to converge on a preferred solution eventually.

So r=me on this for now, looks ok otherwise.

Manishearth commented 7 years ago

bors: r+

frewsxcv commented 7 years ago

I finally got around to looking at this. This is very cool, thanks for doing this @nagisa :100: