HexHive / T-Fuzz

247 stars 39 forks source link

T-Fuzz used wrong code to filter crashing seed? #18

Open zjuchenyuan opened 5 years ago

zjuchenyuan commented 5 years ago

From our experiment to fuzz wav2swf using T-Fuzz, we found an interesting result: The coverage result from the afl queue folder is smaller than provided seed.

So, we investigate this finding, here may be the cause:

https://github.com/HexHive/T-Fuzz/blob/7d150e493237db72c421d423f9a315401cb94e44/tfuzz/executor.py#L39-L48

https://github.com/HexHive/T-Fuzz/blob/7d150e493237db72c421d423f9a315401cb94e44/tfuzz/tfuzz_fuzzer.py#L40-L50

you can see that whenever a CalledProcessError is raised, T-Fuzz will ignore this seed, copy to crashing_seed. And if the whole seed folder all leads to crash, then T-Fuzz will generate a random seed for fuzzing.

However, CalledProcessError will also be raised if the exit code is not 0:

>>> subprocess32.check_output(["/d/p/justafl/4.wav2swf", "-o","/dev/null","crash_seed_12"], timeout=1)
Unsupported bitspersample value: 24
Error: no mp3 soundstream support compiled in.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 638, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess32.CalledProcessError: Command '['/d/p/justafl/4.wav2swf', '-o', '/dev/null', 'crash_seed_12']' returned non-zero exit status 1

I think exit status 1 does not mean a crash, only those be killed by a signal should be.