search

Samsung / UTopia

UT based automated fuzz driver generation
Apache License 2.0
165 stars 26 forks source link

How to test the generated driver? The code coverage I received differs significantly from that in the paper #23

Closed Strke closed 6 months ago

Strke commented 8 months ago

Hello, may I ask how the testing section in the paper is implemented? I put in a lot of effort but still couldn't get the code coverage in the paper, Here is my testing process:

Assuming that a, b, and c here represent three different drivers under libhtp

  1. I first run the libfuzzer file to obtain the profraw files for each driver program, a.profraw, b. profraw, c.profraw
./a  -max_total_time=3600
./b  -max_total_time=3600
./c  -max_total_time=3600
  1. Then use command "llvm-profdata merge" to merge the profraw file

llvm-profdata merge --sparse=true a.profraw b.profraw c.profraw -o foo.profdata

  1. Using command "llvm-cov report" to obtain the code coverage of the entire library file

llvm-cov report -install-profile=foo.profdata -object a b c

But the code coverage results of most of the libraries I obtained are far from those in the paper How can I deal with these problems? Thanks!

yuweol commented 8 months ago

Hello! Sorry for late.

Did you put UTopia generated seed in the queue? UTopia supports automated seed generation, you should put it in a specific directory, for example 'queue'. And then run fuzzer with the queue. ./fuzzer queue. We will add this explanation into README.md file.

Thanks!

sole2 commented 6 months ago

Closing inactive issue. Feel free to reopen if you have more to discuss.

YancyLii commented 4 months ago

I would like to ask, how is coverage information calculated. I first executed all the programs and obtained the. framework files for each program separately. Then, llvm-profdata merge --sparse=true -o default.profdata * Last, llvm-cov show -format=html -output-dir=coverage_report -instr-profile=default.profdata ../../../../../exp/assimp/output/libassimp_profile.a

However, the result I received was an error: libassimp_profile.a: Failed to load coverage: No coverage data found

What is the form of the compilation command for each fuzzer, and why is there a situation where the coverage cannot be found. I'm sorry to ask such a silly question. I'm a beginner in fuzzy testing and I hope to get your help.