google / oss-fuzz-gen

LLM powered fuzzing via OSS-Fuzz.
Apache License 2.0
914 stars 101 forks source link

Less ambiguous and more informative coverage data in report #143

Open DonggeLiu opened 8 months ago

DonggeLiu commented 8 months ago

Current report (e.g., this one) has two main ambiguities regarding coverage data:

  1. The coverage column is 'program counter coverage'. Some readers may assume it is line coverage because the column next to it is 'Line Coverage Diff'.
    • Change the title to 'program counter coverage' (pc coverage is more fine-grained than line coverage).
  2. The link in the 'Line Coverage Diff' is the line coverage of the fuzz target, not the diff line coverage against OSS-Fuzz data.
    • Insert another column for fuzz target line coverage with this link, will need to update Result object.
    • Change the hyperlink of line coverage diff data to the subtracted covreport, which can be written to the result dir with to_file()

Extra: Save the coverage report to result dir too.

DonggeLiu commented 6 months ago

Not related to report UI but related to numbers: On the index page, the line coverage diff should count the union of lines covered by all new fuzz targets, not the maximum of them. That is:

line_coverage = union(target.line_covered for target in all_LLM_generted_fuzz_targets)