MozillaSecurity / lithium

Line-based testcase reducer
Mozilla Public License 2.0
95 stars 25 forks source link

Add ability to always treat the test case as interesting initially #21

Open dkasak opened 7 years ago

dkasak commented 7 years ago

The ability to specify that lithium should treat the test case as interesting initially (before any chunk removals are done, regardless of whether it is actually interesting or not) would be a nice addition.

It would allow lithium to act as a fuzzer, as explained in the Reducers are Fuzzers blog post.

Some use cases:

I'm not sure how to implement this best in lithium, though. Any suggestions? For now, I've just hacked together a non-crashes interestingness test for my personal use. It behaves exactly as crashes, except it forces the test case to be interesting in the initial run. The downside of this approach is that you can't reuse this code with other interestingness tests for which you might want to turn this behaviour on.

nth10sd commented 7 years ago

Having a non-crashes interestingness test would probably be the best way forward.

Note that it is possible to hook in multiple interestingness tests together. e.g. it is possible to use the range interestingness test with the crashes one.

Sample portion of a CLI command:

range 1 20 crashes --timeout=3

will run each iteration 20 times before determining interestingness using the crashes test.