search

AngoraFuzzer / Angora

Angora is a mutation-based fuzzer. The main goal of Angora is to increase branch coverage by solving path constraints without symbolic execution.
Apache License 2.0
916 stars 168 forks source link

ERROR angora::executor::executor > Crash or hang while tracking! -- Timeout, id: 0 #37

Open zhangysh1995 opened 5 years ago

zhangysh1995 commented 5 years ago

I use the following command:

root@f8a44c8e576a:/angora# ./angora_fuzzer -i ../data/base64/fuzzer_input/ -o /root/output/base64_test -t ./base64.tt -- ./base64.fast -d

And it gives:

 INFO  angora::fuzz_main > depot: DepotDir { inputs_dir: "/root/output/base64_test/queue", hangs_dir: "/root/output/base64_test/hangs", crashes_dir: "/root/output/base64_test/crashes", seeds_dir: "../data/base64/fuzzer_input/" }
 INFO  angora::fuzz_main > CommandOpt { id: 0, main: ("./base64.fast", ["-d"]), track: ("./base64.tt", ["-d"]), tmp_dir: "/root/output/base64_test/tmp", out_file: "/root/output/base64_test/tmp/cur_input", forksrv_socket_path: "/root/output/base64_test/tmp/forksrv_socket", track_path: "/root/output/base64_test/tmp/track", is_stdin: true, search_method: Gd, mem_limit: 200, time_limit: 1, is_raw: true, uses_asan: false, ld_library: "$LD_LIBRARY_PATH:/clang+llvm/lib", enable_afl: true, enable_exploitation: true }
 INFO  angora::executor::forksrv > All right -- Init ForkServer /root/output/base64_test/tmp/forksrv_socket_0 successfully!
 ERROR angora::executor::executor > Crash or hang while tracking! -- Timeout,  id: 0
 INFO  angora::depot::sync        > sync       1 file from seeds.
 INFO  angora::bind_cpu           > Found 80 cores.
 INFO  angora::bind_cpu           > Free Cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79]
spinpx commented 5 years ago

base64 -d @@ Please read the document carefully.

zhangysh1995 commented 5 years ago

Why do I need to add @@? I thought it is part of git comments so didn't add it. It is better if you could stress on it.

zhangysh1995 commented 5 years ago

It crashes again with @@, and has a new error with stdin. I have no idea how I should change the directory.

root@f8a44c8e576a:/angora# ./angora_fuzzer  -i ../data/base64/fuzzer_input/ -o /root/output/base64 -j 1 -t ./base64.tt -- ./base64.fast -d @@
 INFO  angora::fuzz_main > depot: DepotDir { inputs_dir: "/root/output/base64/queue", hangs_dir: "/root/output/base64/hangs", crashes_dir: "/root/output/base64/crashes", seeds_dir: "../data/base64/fuzzer_input/" }
 INFO  angora::fuzz_main > CommandOpt { id: 0, main: ("./base64.fast", ["-d", "@@"]), track: ("./base64.tt", ["-d", "@@"]), tmp_dir: "/root/output/base64/tmp", out_file: "/root/output/base64/tmp/cur_input", forksrv_socket_path: "/root/output/base64/tmp/forksrv_socket", track_path: "/root/output/base64/tmp/track", is_stdin: false, search_method: Gd, mem_limit: 200, time_limit: 1, is_raw: true, uses_asan: false, ld_library: "$LD_LIBRARY_PATH:/clang+llvm/lib", enable_afl: true, enable_exploitation: true }
 INFO  angora::executor::forksrv > All right -- Init ForkServer /root/output/base64/tmp/forksrv_socket_0 successfully!
 ERROR angora::executor::executor > Crash or hang while tracking! -- Timeout,  id: 0
 INFO  angora::depot::sync        > sync       1 file from seeds.
 INFO  angora::bind_cpu           > Found 80 cores.
 INFO  angora::bind_cpu           > Free Cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79]

Here is the full bash log: angora_crashes_in_container.txt

Could I also know how to control CPU core numbers?

spinpx commented 5 years ago

Thanks for your feedback.

It crashes again with @@

Could you provide the seeds for me for reproduction? Please try to use a small input as seeds in the evaluation. The reason may be: Your seed is too large, it takes much time to do taint analysis.

so you can:

Why do I need to add @@? I thought it is part of git comments so didn't add it. It is better if you could stress on it.

We use @@ as input file following AFL's usage.

zhangysh1995 commented 5 years ago

I used the standard fuzzer_input folder provided by LAVA-M, and it is located in xxx/lava_corpus/LAVA-M/base64/fuzzer_input (the dataset is downloaded with this link in this repo).

Here are other environment details:

yushanzhang@ubuntu:~$ clang --version
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/yushanzhang/local/clang+llvm/bin

yushanzhang@ubuntu:~$ gclang --version
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/yushanzhang/local/clang+llvm/bin

yushanzhang@ubuntu:~$ rustup --version
rustup 1.16.0 (beab5ac2b 2018-12-06)

yushanzhang@ubuntu:~$ cargo --version
cargo 1.32.0 (8610973aa 2019-01-02)

yushanzhang@ubuntu:~$ uname -r
4.4.0-93-generic
spinpx commented 5 years ago

The fuzzer_input is too long to taint tracking. You can try "Hello world" as the seed input.

zhangysh1995 commented 5 years ago

And my compiled base64.tt has segmenationt fault error. I followed all the same commands to compile this binary.

yushanzhang@ubuntu:~/fuzzing/angora$ USE_TRACK=1 bin/angora-clang base64.bc -o base64.tt
angora-llvm-pass
[+] Track Mode.
ModName: base64.bc -- 344208029
Input is LLVM bitcode
[+] Max constraint id is 1266
yushanzhang@ubuntu:~/fuzzing/angora$ ./base64.tt
Segmentation fault
yushanzhang@ubuntu:~/fuzzing/angora$ ./base64.tt -d "hello"
Segmentation fault
yushanzhang@ubuntu:~/fuzzing/angora$ ./base64.tt -d ../lava-m/base64/fuzzer_input/rand.b64
Segmentation fault

However, the base64.fast and base64 could work.

spinpx commented 5 years ago

Sorry. I can't reproduce your issue. Can you provide the stack backtrace for the fault, or the executable, or any other information in detail?

zhangysh1995 commented 5 years ago

Seems the crashpoint is part of the sanitizer check:

(gdb) r
Starting program: /home/yushanzhang/fuzzing/angora/base64.tt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
__sanitizer::internal_mmap (addr=0x10000, length=70368744112128, prot=<optimized out>, flags=<optimized out>,
    fd=<optimized out>, offset=0) at sanitizer_common/sanitizer_linux.cc:123
123       return internal_syscall(SYSCALL(mmap), (uptr)addr, length, prot, flags, fd,
(gdb) b 122
Breakpoint 2 at 0x10004c041: file sanitizer_common/sanitizer_linux.cc, line 122.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/yushanzhang/fuzzing/angora/base64.tt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 2, __sanitizer::internal_mmap (addr=0x0, length=8192, prot=<optimized out>, flags=<optimized out>,
    fd=<optimized out>, offset=0) at sanitizer_common/sanitizer_linux.cc:123
123       return internal_syscall(SYSCALL(mmap), (uptr)addr, length, prot, flags, fd,
(gdb) n
__sanitizer::MmapOrDie (size=8192, mem_type=0x100085131 "Allocate",
    raw_report=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
    at sanitizer_common/sanitizer_posix.cc:131
131       if (internal_iserror(res, &reserrno))
(gdb) n
133       IncreaseTotalMmap(size);
(gdb) n
134       return (void *)res;
(gdb) n
__sanitizer::LowLevelAllocator::Allocate (this=0x1008c6760 <__sanitizer::FlagParser::Alloc>, size=4800)
    at sanitizer_common/sanitizer_allocator.cc:187
187         allocated_current_ =
(gdb) n
189         allocated_end_ = allocated_current_ + size_to_allocate;
(gdb) n
190         if (low_level_alloc_callback) {
(gdb) n
195       CHECK(allocated_end_ - allocated_current_ >= (sptr)size);
(gdb) n
197       allocated_current_ += size;
(gdb) n
198       return res;
(gdb) n
__sanitizer::FlagParser::FlagParser (this=0x7fffffffe2d8) at sanitizer_common/sanitizer_flag_parser.cc:169
169     }
(gdb) n
InitializeFlags () at rt/dfsan.cc:226
226       RegisterCommonFlags(&parser);
(gdb) n

Breakpoint 2, __sanitizer::internal_mmap (addr=0x0, length=4096, prot=<optimized out>, flags=<optimized out>,
    fd=<optimized out>, offset=0) at sanitizer_common/sanitizer_linux.cc:123
123       return internal_syscall(SYSCALL(mmap), (uptr)addr, length, prot, flags, fd,

The context of line 123 in file llvm_mode/dfsan/sanitizer_common/sanitizer_linux.cc:


namespace __sanitizer {

#if SANITIZER_LINUX && defined(__x86_64__)
#include "sanitizer_syscall_linux_x86_64.inc"
#elif SANITIZER_LINUX && defined(__aarch64__)
#include "sanitizer_syscall_linux_aarch64.inc"
#else
#include "sanitizer_syscall_generic.inc"
#endif

// --------------- sanitizer_libc.h
#if !SANITIZER_S390
uptr internal_mmap(void *addr, uptr length, int prot, int flags, int fd,
                   OFF_T offset) {
#if SANITIZER_FREEBSD || SANITIZER_LINUX_USES_64BIT_SYSCALLS
  return internal_syscall(SYSCALL(mmap), (uptr)addr, length, prot, flags, fd,
                          offset);
#else
  // mmap2 specifies file offset in 4096-byte units.
  CHECK(IsAligned(offset, 4096));
  return internal_syscall(SYSCALL(mmap2), addr, length, prot, flags, fd,
                          offset / 4096);
#endif
}
#endif // !SANITIZER_S390

Seems this is an internal error of the LLVM sanitizer.

spinpx commented 5 years ago

Hi @zhangysh1995 . Thanks for your feedback. Sorry I have no idea of fixing it now. We have tested angora both in ubuntu16.04&18.04 64bit, so I assume the problem is not caused by the distribution you are using. Please try to ensure you did not limit the memory for applications since DFSan needs huge memory. Also, does the version built in docker works?

You can test if angora is built successfully by : https://github.com/AngoraFuzzer/Angora#test

zhangysh1995 commented 5 years ago

@spinpx 

yushanzhang@ubuntu:~$ ulimit
unlimited

I ran the tests seems no crash but no output from the executor:

yushanzhang@ubuntu:~/fuzzing/angora/tests$ ./test.sh mini
+ BUILD_TYPE=debug
+ num_jobs=1
+ sync_afl=
+ LOG_TYPE=angora
+ [ ! -z ]
+ envs=RUST_BACKTRACE=1 RUST_LOG=angora
+ fuzzer=../target/debug/fuzzer
+ input=./input
+ output=./output
+ [ 1 -ne 1 ]
+ [ -d mini ]
+ rm -rf ./output
+ name=mini
+ echo Compile...
Compile...
+ target=mini/mini
+ rm -f mini/mini.fast mini/mini.cmp mini/mini.taint
+ bin_dir=../bin/
+ ANGORA_USE_ASAN=1 USE_FAST=1 ../bin//angora-clang mini/mini.c -lz -o mini/mini.fast
angora-llvm-pass
[+] Fast Mode.
ModName: mini/mini.c -- 167608937
+ USE_TRACK=1 ../bin//angora-clang mini/mini.c -lz -o mini/mini.taint
angora-llvm-pass
[+] Track Mode.
ModName: mini/mini.c -- 167608937
+ echo Compile Done..
Compile Done..
+ args_file=./mini/args
+ [ ! -f ./mini/args ]
+ cat ./mini/args
+ args=@@
+ cmd=RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1
+ cmd=RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1 -t mini/mini.taint  -- mini/mini.fast @@
+ echo run: RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1 -t mini/mini.taint  -- mini/mini.fast @@
run: RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1 -t mini/mini.taint  -- mini/mini.fast @@
+ eval RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1 -t mini/mini.taint -- mini/mini.fast @@
+ RUST_BACKTRACE=1 RUST_LOG=angora ../target/debug/fuzzer -M 0 -A -i ./input -o ./output -j 1 -t mini/mini.taint -- mini/mini.fast @@
 INFO  angora::fuzz_main > depot: DepotDir { inputs_dir: "./output/queue", hangs_dir: "./output/hangs", crashes_dir: "./output/crashes", seeds_dir: "./input" }
 INFO  angora::fuzz_main > CommandOpt { id: 0, main: ("mini/mini.fast", ["@@"]), track: ("mini/mini.taint", ["@@"]), tmp_dir: "./output/tmp", out_file: "./output/tmp/cur_input", forksrv_socket_path: "./output/tmp/forksrv_socket", track_path: "./output/tmp/track", is_stdin: false, search_method: Gd, mem_limit: 0, time_limit: 1, is_raw: true, uses_asan: true, ld_library: "$LD_LIBRARY_PATH:/home/yushanzhang/local/clang+llvm/lib", enable_afl: false, enable_exploitation: true }
 DEBUG angora::executor::forksrv > socket_path: "./output/tmp/forksrv_socket_0"

I also tried the docker container, but seems base64.tt just start and has no response to any input. And I has to stop the container rather than sending signal to stop it.

Would it be possible that segmentation fault and no response errors are caused by gclang?