MozillaSecurity / lithium

Line-based testcase reducer
Mozilla Public License 2.0
94 stars 25 forks source link

Skip testing with empty testcase when large chunks are removed #57

Closed tysmith closed 6 years ago

tysmith commented 6 years ago

Currently if we have a testcase of size 4246 and our first reduction pass of size 4096 is successful we then have a testcase of size 150. Then we will try to remove the remaining part and then reduce the chunk size by 1/2. The problem here is we will then try with 2048, 1024, 512, 256, before getting to 128. This means we will test with an empty testcase 5 times. This can be optimized. PR to come.

Here is an example log of the issue:

Intermediate files will be stored in tmp129/. The original testcase has 4246 lines. Checking that the original testcase is 'interesting'... Interesting: Assertion failure: blah... blah... blah... Updating poll timeout to: 10 Updating max timeout to: 10 Interesting: Assertion failure: blah... blah... blah... Removing a chunk of size 4096 starting at 0 of 4246 was a successful reduction :) Uninteresting: no failure detected Removing a chunk of size 4096 starting at 0 of 150 made the file 'uninteresting'.

Halving chunk size to 2048 Uninteresting: no failure detected Removing a chunk of size 2048 starting at 0 of 150 made the file 'uninteresting'.

Halving chunk size to 1024 Uninteresting: no failure detected Removing a chunk of size 1024 starting at 0 of 150 made the file 'uninteresting'.

Halving chunk size to 512 Uninteresting: no failure detected Removing a chunk of size 512 starting at 0 of 150 made the file 'uninteresting'.

Halving chunk size to 256 Uninteresting: no failure detected Removing a chunk of size 256 starting at 0 of 150 made the file 'uninteresting'.

Halving chunk size to 128 Uninteresting: no failure detected Removing a chunk of size 128 starting at 0 of 150 made the file 'uninteresting'. Interesting: Assertion failure: blah... blah... blah... Removing a chunk of size 128 starting at 128 of 150 was a successful reduction :)