swtv-kaist / cs458-spring24

7 stars 0 forks source link

[HW6] What is the "branch coverage" reported by AFL++? #36

Closed tncks0121 closed 5 months ago

tncks0121 commented 5 months ago

In my AFL++ execution, the "map density" (or bitmap_cvg) is significantly smaller than "Taken at least once" in gcov. For example, bitmap_cvg is like 5% when "Taken at least once" is like 60%.

From the lecture, I think I have to measure "map density" for branch coverage. The AFL++ documentation also says "bitmap_cvg - percentage of edge coverage found in the map so far". I understand that AFL++ measures approximate coverage (by using buckets), but the value is very different.

From a previous issue here, it seems AFL++ coverage is not that different from gcov coverage.

So I think I am using the wrong metric. Is it not intended to use bitmap_cvg for measuring branch coverage?

3-24 commented 5 months ago

First, The bitmap_cvg in the AFL++ status file corresponds to the branch coverage reported by AFL++. It is expected to give similar value as gcov's branch coverage, as I answered in the last year's discussion. At this point, I don't know why your result is far different from the gcov's result. Can you give more information about how you measured and collected coverages from both AFL++ and gcov?

As a hint, you can also access the AFL++'s time-coverage plot in plot_data file. The map_size entry corresponds to it and it is identical to bitmap_cvg.

tncks0121 commented 5 months ago

Thanks for the clarification. I compiled grep with afl-gcc-fast. Seems like I have to compile with afl-clang-fast.

3-24 commented 5 months ago

Wow, I didn't realize there was such an issue. By default, I assumed using afl-clang-fast or afl-clang-lto as you said.