google / honggfuzz

Security oriented software fuzzer. Supports evolutionary, feedback-driven fuzzing based on code coverage (SW and HW based)
https://honggfuzz.dev
Apache License 2.0
3.07k stars 515 forks source link

Coverage never reaches 100% #510

Open AntwanEmil opened 8 months ago

AntwanEmil commented 8 months ago

After trying a simple LLVM persistent fuzzing example:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <inttypes.h>

int func2(size_t len){
        int i=0;
        if(len>0){
                i = 5;
        }
        return 1;
}

extern int LLVMFuzzerTestOneInput(uint8_t *buf, size_t len){
        int i=func2(len);
        return 0;
}

using this command to cover the edges (default is edge coverage): $hfuzz-clang -g -fsanitize=address -fsanitize-coverage=edge edge.c -o edge

Hongfuzz output always shows Coverage : edge: 4/5 [80%] pc: 0 cmp: 64 which is an edge more than expected, and it is never reached.

Also after trying to cover number of functions: hfuzz-clang -g -fsanitize=address -fsanitize-coverage=func edge.c -o edge
output shows Coverage : edge: 2/3 [66%] pc: 0 cmp: 64


After doing some simple search, I think the missing function here is the main() function used by llvm to call the LLVMFuzzerTestOneInput() function. Is there any way I can exclude this from coverage to reach 100% ?

robertswiecki commented 8 months ago

The reason is using initial guardNo=1 here https://github.com/google/honggfuzz/blob/348a47213919f14b9453e89a663b1515369bd9a2/libhfuzz/instrument.c#L259

IIRC it's required by clang instrumentation API (ie. value > 0), otherwise it'll be skipped when calling back into honggfuzz instrumentation code.

I believe the proper fix is to fix display.c and lower the displayed number of total edges by 1. E.g. even if there's no instrumentation, display.c still shows it as 1

$ gcc edge.c -o edge ~/src/honggfuzz/libhfuzz/libhfuzz.a ~/src/honggfuzz/libhfcommon/libhfcommon.a
------------------------[  0 days 00 hrs 03 mins 25 secs ]----------------------
  Iterations : 11,423,715 [11.42M]
  Mode [3/3] : Feedback Driven Mode
      Target : ./edge
     Threads : 1, CPUs: 12, CPU%: 82% [6%/CPU]
       Speed : 53,637/sec [avg: 55,725]
     Crashes : 0 [unique: 0, blocklist: 0, verified: 0]
    Timeouts : 0 [1 sec]
 Corpus Size : 1, max: 8,192 bytes, init: 3 files
  Cov Update : 0 days 00 hrs 03 mins 25 secs ago
    Coverage : edge: 0/1 [0%] pc: 0 cmp: 0
---------------------------------- [ LOGS ] ------------------/ honggfuzz 2.6 /-
AntwanEmil commented 8 months ago

Easy fix would be:

diff --git a/display.c b/display.c
index 54644acb..0c6506ad 100644
--- a/display.c
+++ b/display.c
@@ -415,7 +415,7 @@ void display_display(honggfuzz_t* hfuzz) {
         uint64_t softCntPc   = ATOMIC_GET(hfuzz->feedback.hwCnts.softCntPc);
         uint64_t softCntEdge = ATOMIC_GET(hfuzz->feedback.hwCnts.softCntEdge);
         uint64_t softCntCmp  = ATOMIC_GET(hfuzz->feedback.hwCnts.softCntCmp);
-        uint64_t guardNb     = ATOMIC_GET(hfuzz->feedback.covFeedbackMap->guardNb);
+        uint64_t guardNb     = ATOMIC_GET(hfuzz->feedback.covFeedbackMap->guardNb)-1;
         display_put(" edge: " ESC_BOLD "%" _HF_NONMON_SEP PRIu64 ESC_RESET "/"
                     "%" _HF_NONMON_SEP PRIu64 " [%" PRId64 "%%]",
             softCntEdge, guardNb, guardNb ? ((softCntEdge * 100) / guardNb) : 0);