intel / tsffs

A snapshotting, coverage-guided fuzzer for software (UEFI, Kernel, firmware, BIOS) built on SIMICS
https://intel.github.io/tsffs/
Apache License 2.0
273 stars 17 forks source link

Similar testcases while fuzzing platform bios #101

Open today574 opened 3 weeks ago

today574 commented 3 weeks ago

The fuzzing process for example 5.3 "Fuzzing a platform BIOS" started successfully: tsffs_starts

I expected to see the bugs added in example after some time. I left the fuzzer running for a while and the logs are as follows: logs_2_0

The first thing that confused me is exec/sec: 0.000. I tried to use different data in corpus, but specifically in this case I used 7 different bmp images. So the second thing that is confusing is that the fuzzer used only two input images (only 2 were created in the corpus) and then nothing happened over time new_corpus

After that, I stopped the fuzzer and started it again (I did not delete those 2 pictures in the corpus). And in this case exec/sec: is not 0.000 second_time_start_fuzz

next, I repeated the previous iterations but at the logging level 4. I left only 7 test bmp images and run a fuzzer with a logging level 4. As you can see, testcases are the same logs4_firststart

After that, I restarted the fuzzer (there are 2 additional images in the case from the previous start). In the following logs, you can see that the size of the testcases is different, but the data is the same: logs4_secondstart

And as a result, it is not possible to find the added bugs, the solutions folder is empty

novafacing commented 3 weeks ago

I'm not sure if this is a bug, the fact that the testcase sizes are different necessarily means they are different -- the log just displays the first 32 bytes otherwise we'd be polluting the terminal. It also might explain why the bug might not be found, because it is only triggered by a strangely formed (usually small) BMP file.

The fuzzer only keeping two inputs from the corpus is expected. This means those two files triggered different coverage, while the rest triggered the same coverage. This is a fuzzing algorithm optimization to keep mutating only testcases which do something new.

The 0.00 exec/s is strange though. I'll look into it.