search

evanmak / savior-source

source code for savior fuzzer
Apache License 2.0
126 stars 27 forks source link

AFL file patterns #1

Closed cponcelets closed 4 years ago

cponcelets commented 4 years ago

Dear Savior team,

There is a misuse of AFL regarding the test case file names. AFL is computed using the SIMPLE_FILES variable making the fuzzer to print files using id_ patterns. However, Converter is translating Klee test cases using the standard id: patterns which prevents AFL from importing the test cases.

You might want to choose one of the two patterns.

Thank you for this interesting tool, Best

evanmak commented 4 years ago

Thanks for raising the concern, in the AFL config.h header, SIMPLE_FILES is commented out. So the two should be using the same naming schema.

cponcelets commented 4 years ago

Yes, I also expected that. Unfortunately, the variable is passed into the Makefile command line which defines SIMPLE_FILES.

evanmak commented 4 years ago

Nice catch! this is not standard AFL configuration in the Makefile. will update. Just curious, how did you find this issue?

cponcelets commented 4 years ago

Well, I figured out that SAVIOR had a strange behavior on some benchmarks and did not cover simple paths.

I thought first that it was my configuration but nothing changed with different ones. Moreover, the number of imported test cases by AFL was always very low. Here, I discovered the differences between afl and klee test cases so during a SAVIOR fuzzing I renamed the test cases in one of the klee folder to match the AFL pattern and that magically boosted AFL counters!

After that I just have to parse AFL code and understand how they were managing that file pattern (I did not know about the simple_files feature). I figured out that it was a compile-time variable, so after checking into the config file and decided to check the Makefile, and here we are :)

evanmak commented 4 years ago

Thanks for the analysis. This typo could be introduced when I cleaned up the code. It is still strange because iirc if the naming mismatches, there shouldn't be any import by AFL.

If this is addressed, would you mind closing the issue? Happy fuzzing : )

cponcelets commented 4 years ago

Yes, of course. Thank you for your quick answer.

Notice that you want to update afl-cov which also use 'id'. This prevents from using coverage based oracles (klee crash since it cannot find `id` files). May you fix it also or should I open another issue? (only updating the file with up-to-date afl-cov repo will work)

Thanks again, Happy fuzzing too.