Open dkasak opened 7 years ago
Having a non-crashes
interestingness test would probably be the best way forward.
Note that it is possible to hook in multiple interestingness tests together. e.g. it is possible to use the range
interestingness test with the crashes
one.
Sample portion of a CLI command:
range 1 20 crashes --timeout=3
will run each iteration 20 times before determining interestingness using the crashes
test.
The ability to specify that lithium should treat the test case as interesting initially (before any chunk removals are done, regardless of whether it is actually interesting or not) would be a nice addition.
It would allow lithium to act as a fuzzer, as explained in the Reducers are Fuzzers blog post.
Some use cases:
lithium crashes
run while forcing it to treat the case as interesting.I'm not sure how to implement this best in lithium, though. Any suggestions? For now, I've just hacked together a
non-crashes
interestingness test for my personal use. It behaves exactly ascrashes
, except it forces the test case to be interesting in the initial run. The downside of this approach is that you can't reuse this code with other interestingness tests for which you might want to turn this behaviour on.