search

BurntSushi / jiff

A date-time library for Rust that encourages you to jump into the pit of success.
The Unlicense
1.76k stars 33 forks source link

add fuzz testing #14

Open BurntSushi opened 4 months ago

BurntSushi commented 4 months ago

We should be able to do some fuzz testing. The minimal thing we can do is to throw data at the datetime and span parsers to ensure they don't panic.

sigaloid commented 3 months ago

In my experience, a mutation-based fuzzer like AFL++/afl.rs converges on panics much more quickly when given a corpus to start out from, since they start out with examples that provide way better coverage. Something like this, except with some additional handling to generate test case files from src/tz/mod.rs: https://github.com/MaxOhn/rosu-map/pull/3/

I'd be happy to submit a PR if this is roughly what you're looking for!

BurntSushi commented 3 months ago

I don't have a ton of domain expertise with fuzzing. I just know some of the basics. Is that setup compatible with OSS-fuzz? I'd like to make sure that whatever I do, it can run inside of OSS-fuzz.

BurntSushi commented 3 months ago

cc @AddisonCrump

sigaloid commented 3 months ago

According to OSS-fuzz, no - while they say they support AFL++ as an engine on the home page, they mention on the actual Rust integration page that they only support cargo fuzz, which is built around libFuzzer. However, similar methods can be used to seed libFuzzer with a corpus, so some code to extract the test cases out would help improve fuzzing coverage.

addisoncrump commented 3 months ago

Given that most of the issues that we would want to look for in this kind of project are correctness issues, any fuzzer capable of finding bugs beyond low hanging fruit would need a very strong grammar and a good oracle for correctness.

That's not to say that I don't think this would be valuable. For initial testing for crashes, this would certainly be valuable :+1: Prefer cargo-fuzz, though, as we can integrate this with other fuzzers (e.g., libafl) more easily.

BurntSushi commented 3 months ago

Yeah I think a good start here would be fuzzing the parsers. There's a fair bit of surface area there.

addisoncrump commented 3 months ago

I'll do a quick review of parse functions and make some simple harnesses.