Open duncan-bayne opened 3 years ago
This indeed looks weird. Let me ask you a dumb question please. Are you sure you're passing the right (i.e. instrumented, just built) binary to the fuzzer?
I see the following build command:
/usr/local/bin/afl-gcc -g -O2 -o munpack unixunpk.o decode.o uudecode.o codes.o unixos.o part.o xmalloc.o md5c.o
which states -o munpack
, but your fuzzing command seems to be using /usr/home/duncan/code/mpack/mpack
. Is it the same binary?
Yes, very sure I'm afraid. The working directory for make clean all
in this case is /usr/home/duncan/code/mpack/
. And it's definitely using afl-gcc
for compilation; if I look in the generated Makefile I see:
CC = /usr/local/bin/afl-gcc
... and the logs show the correct compiler is being invoked:
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c unixpk.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c encode.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c codes.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c magic.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c unixos.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c xmalloc.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c md5c.c
/usr/local/bin/afl-gcc -g -O2 -o mpack unixpk.o encode.o codes.o magic.o unixos.o xmalloc.o md5c.o
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c unixunpk.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c decode.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c uudecode.c
/usr/local/bin/afl-gcc -DPACKAGE=\"mpack\" -DVERSION=\"1.6\" -DHAVE_DN_EXPAND=1 -DHAVE_STRCHR=1 -DHAVE_STRCASECMP=1 -DHAVE_GETOPT=1 -I. -I. -g -O2 -c part.c
/usr/local/bin/afl-gcc -g -O2 -o munpack unixunpk.o decode.o uudecode.o codes.o unixos.o part.o xmalloc.o md5c.o
Okay, I've created a much simpler test case that exhibits the same behaviour.
I take the following C program:
#include <stdio.h>
#include <stdlib.h>
int main(void) {
printf("Hello, world!\n");
return EXIT_SUCCESS;
}
... compile it with ...
$ afl-gcc test.c -Wall --ansi --pedantic -o ./test
... then run it with afl-fuzz
, it produces the same error:
$ afl-fuzz -i /usr/home/duncan/tests/afl/testcases -o /usr/home/duncan/tests/afl/output ./test -s Test -o /usr/home/duncan/tmp/out @@
afl-fuzz 2.52b by <lcamtuf@google.com>
[+] You have 8 CPU cores and 1 runnable tasks (utilization: 12%).
[+] Try parallel jobs - see /usr/local/share/doc/afl/parallel_fuzzing.txt.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning '/usr/home/duncan/tests/afl/testcases'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[-] Looks like the target binary is not instrumented! The fuzzer depends on
compile-time instrumentation to isolate interesting test cases while
mutating the input data. For more information, and for tips on how to
instrument binaries, please see /usr/local/share/doc/afl/README.
When source code is not available, you may be able to leverage QEMU
mode support. Consult the README for tips on how to enable this.
(It is also possible to use afl-fuzz as a traditional, "dumb" fuzzer.
For that, you can use the -n option - but expect much worse results.)
B
[-] PROGRAM ABORT : No instrumentation detected
Location : check_binary(), afl-fuzz.c:6920
So I'm pretty sure that either AFL is entirely broken on my system, or I'm doing something entirely dumb. Given this is my first time using fuzzers in general and AFL in particular, I'm betting on the latter.
@duncan-bayne I see that you're using an old version of AFL, maybe that's the reason. I've just updated my checkout of this repository and ran the following commands:
1) compile AFL
$ make
[*] Checking for the ability to compile x86 code...
[+] Everything seems to be working, ready to compile.
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-gcc.c -o afl-gcc -ldl
set -e; for i in afl-g++ afl-clang afl-clang++; do ln -sf afl-gcc $i; done
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-fuzz.c -o afl-fuzz -ldl
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-showmap.c -o afl-showmap -ldl
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-tmin.c -o afl-tmin -ldl
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-gotcpu.c -o afl-gotcpu -ldl
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-analyze.c -o afl-analyze -ldl
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-as.c -o afl-as -ldl
ln -sf afl-as as
[*] Testing the CC wrapper and instrumentation output...
unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-gcc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" test-instr.c -o test-instr -ldl
./afl-showmap -m none -q -o .test-instr0 ./test-instr < /dev/null
echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr
[+] All right, the instrumentation seems to be working!
[+] All done! Be sure to review README - it's pretty short and useful.
NO
2) compile your example
$ ./afl-gcc test.c -Wall --ansi --pedantic -o ./test
afl-cc 2.57b by <lcamtuf@google.com>
afl-as 2.57b by <lcamtuf@google.com>
[+] Instrumented 1 locations (64-bit, non-hardened mode, ratio 100%).
3) launch fuzzing:
$ AFL_SKIP_CPUFREQ=1 ./afl-fuzz -i testcases/ -o /tmp/afl ./test -s Test -o /tmp/out
afl-fuzz 2.57b by <lcamtuf@google.com>
[+] You have 8 CPU cores and 2 runnable tasks (utilization: 25%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #0.
[*] Checking core_pattern...
[*] Setting up output directories...
[*] Scanning 'testcases/'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,orig:README.testcases'...
[*] Spinning up the fork server...
[+] All right - fork server is up.
len = 828, map size = 1, exec speed = 354 us
[+] All test cases processed.
[+] Here are some useful stats:
Test case count : 1 favored, 0 variable, 1 total
Bitmap range : 1 to 1 bits (average: 1.00 bits)
Exec timing : 354 to 354 us (average: 354 us)
[*] No -t option specified, so I'll use exec timeout of 20 ms.
[+] All set and ready to roll!
american fuzzy lop 2.57b (test)
┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐
│ run time : 0 days, 0 hrs, 0 min, 1 sec │ cycles done : 23 │
│ last new path : none yet (odd, check syntax!) │ total paths : 1 │
│ last uniq crash : none seen yet │ uniq crashes : 0 │
│ last uniq hang : none seen yet │ uniq hangs : 0 │
├─ cycle progress ────────────────────┬─ map coverage ─┴───────────────────────┤
│ now processing : 0 (0.00%) │ map density : 0.00% / 0.00% │
│ paths timed out : 0 (0.00%) │ count coverage : 1.00 bits/tuple │
├─ stage progress ────────────────────┼─ findings in depth ────────────────────┤
│ now trying : havoc │ favored paths : 1 (100.00%) │
│ stage execs : 158/256 (61.72%) │ new edges on : 1 (100.00%) │
│ total execs : 7315 │ total crashes : 0 (0 unique) │
│ exec speed : 4013/sec │ total tmouts : 0 (0 unique) │
├─ fuzzing strategy yields ───────────┴───────────────┬─ path geometry ────────┤
│ bit flips : 0/32, 0/31, 0/29 │ levels : 1 │
│ byte flips : 0/4, 0/3, 0/1 │ pending : 0 │
│ arithmetics : 0/224, 0/0, 0/0 │ pend fav : 0 │
│ known ints : 0/24, 0/84, 0/44 │ own finds : 0 │
│ dictionary : 0/0, 0/0, 0/0 │ imported : n/a │
│ havoc : 0/6656, 0/0 │ stability : 100.00% │
│ trim : 99.52%/16, 0.00% ├────────────────────────┘
^C────────────────────────────────────────────────────┘ [cpu000: 30%]
+++ Testing aborted by user +++
[+] We're done here. Have a nice day!
Please give it a try with the latest version :)
I'm afraid the problem persists with the latest version, compiled from source. The entire repro process (with a few irrelevant bits snipped):
➜ ~ cd ~/code
➜ code git clone https://github.com/google/AFL
Cloning into 'AFL'...
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 527 (delta 1), reused 2 (delta 0), pack-reused 519
Receiving objects: 100% (527/527), 1007.20 KiB | 868.00 KiB/s, done.
Resolving deltas: 100% (189/189), done.
➜ code cd AFL
➜ AFL git:(master) gmake
[*] Checking for the ability to compile x86 code...
[+] Everything seems to be working, ready to compile.
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-gcc.c -o afl-gcc
set -e; for i in afl-g++ afl-clang afl-clang++; do ln -sf afl-gcc $i; done
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-fuzz.c -o afl-fuzz
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-showmap.c -o afl-showmap
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-tmin.c -o afl-tmin
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-gotcpu.c -o afl-gotcpu
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-analyze.c -o afl-analyze
cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" afl-as.c -o afl-as
ln -sf afl-as as
[*] Testing the CC wrapper and instrumentation output...
unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-clang -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DDOC_PATH=\"/usr/local/share/doc/afl\" -DBIN_PATH=\"/usr/local/bin\" test-instr.c -o test-instr
./afl-showmap -m none -q -o .test-instr0 ./test-instr < /dev/null
echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr
[+] All right, the instrumentation seems to be working!
[+] LLVM users: see llvm_mode/README.llvm for a faster alternative to afl-gcc.
[+] All done! Be sure to review README - it's pretty short and useful.
NOTE: If you can read this, your terminal probably uses white background.
This will make the UI hard to read. See docs/status_screen.txt for advice.
➜ AFL git:(master) ./afl-gcc --version
afl-cc 2.57b by <lcamtuf@google.com>
gcc (FreeBSD Ports Collection) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
➜ AFL git:(master) ./afl-fuzz --version
afl-fuzz 2.57b by <lcamtuf@google.com>
➜ AFL git:(master) cd ~/tmp/fuzz
➜ fuzz git:(master) ✗ ls -l
total 4
-rw-r--r-- 1 duncan duncan 111 13 Oct 21:22 test.c
➜ fuzz git:(master) ✗ /usr/home/duncan/code/AFL/afl-gcc test.c -Wall --ansi --pedantic -o ./test
afl-cc 2.57b by <lcamtuf@google.com>
➜ fuzz git:(master) ✗ ls -l
total 20
-rwxr-xr-x 1 duncan duncan 16064 17 Oct 11:59 test
-rw-r--r-- 1 duncan duncan 111 13 Oct 21:22 test.c
➜ fuzz git:(master) ✗ /usr/home/duncan/code/AFL/afl-fuzz -i /usr/home/duncan/tests/afl/testcases -o /usr/home/duncan/tests/afl/output ./test -s Test -o /usr/home/duncan/tmp/out @@
afl-fuzz 2.57b by <lcamtuf@google.com>
[+] You have 8 CPU cores and 1 runnable tasks (utilization: 12%).
[+] Try parallel jobs - see /usr/local/share/doc/afl/parallel_fuzzing.txt.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning '/usr/home/duncan/tests/afl/testcases'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[-] Looks like the target binary is not instrumented! The fuzzer depends on
compile-time instrumentation to isolate interesting test cases while
mutating the input data. For more information, and for tips on how to
instrument binaries, please see /usr/local/share/doc/afl/README.
When source code is not available, you may be able to leverage QEMU
mode support. Consult the README for tips on how to enable this.
(It is also possible to use afl-fuzz as a traditional, "dumb" fuzzer.
For that, you can use the -n option - but expect much worse results.)
[-] PROGRAM ABORT : No instrumentation detected
Location : check_binary(), afl-fuzz.c:6989
Note that the self-test appears to work, but the resultant binary is still not detected as instrumented.
Could we please re-open this issue?
Do you have afl-as
file built as well? In my afl-gcc output I saw the following:
$ ./afl-gcc test.c -Wall --ansi --pedantic -o ./test
afl-cc 2.57b by <lcamtuf@google.com>
afl-as 2.57b by <lcamtuf@google.com>
[+] Instrumented 1 locations (64-bit, non-hardened mode, ratio 100%).
while yours is just:
➜ AFL git:(master) ./afl-fuzz --version
afl-fuzz 2.57b by <lcamtuf@google.com>
The gcc version seems to be fine, e.g. mine:
$ ./afl-gcc --version
afl-cc 2.57b by <lcamtuf@google.com>
gcc (Debian 9.3.0-15) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Afraid so. This is what got built in /usr/home/duncan/code/AFL
:
-rwxr-xr-x 1 duncan duncan 11391 17 Oct 11:58 ./afl-cmin
-rwxr-xr-x 1 duncan duncan 4912 17 Oct 11:58 ./afl-plot
-rwxr-xr-x 1 duncan duncan 3711 17 Oct 11:58 ./afl-whatsup
-rwxr-xr-x 1 duncan duncan 33752 17 Oct 11:58 ./afl-gcc
-rwxr-xr-x 1 duncan duncan 334912 17 Oct 11:58 ./afl-fuzz
-rwxr-xr-x 1 duncan duncan 58648 17 Oct 11:58 ./afl-showmap
-rwxr-xr-x 1 duncan duncan 74816 17 Oct 11:58 ./afl-tmin
-rwxr-xr-x 1 duncan duncan 25800 17 Oct 11:58 ./afl-gotcpu
-rwxr-xr-x 1 duncan duncan 64032 17 Oct 11:58 ./afl-analyze
-rwxr-xr-x 1 duncan duncan 54352 17 Oct 11:58 ./afl-as
Thinking perhaps it couldn't find afl-as
, I tried the following:
/usr/home/duncan/code/AFL
to PATH. test.c.
to /usr/home/duncan/code/AFL
, and invoking afl-gcc
and afl-fuzz
from there. Unfortunately, neither approach fixed the problem.
If it'd help, I can give you SSH access to a FreeBSD box for testing / debugging purposes.
Anyone interested in access to a FreeBSD box as suggested above?
I'm trying to fuzz an unmodified fork of the mpack source.
expected behaviour
After building from source using
afl-gcc
, I'd expect to be able to fuzz the binary.actual behaviour
I get an error
Looks like the target binary is not instrumented!
.versions of things
example session
a cute picture of a
rabbithare