search

AngoraFuzzer / Angora

Angora is a mutation-based fuzzer. The main goal of Angora is to increase branch coverage by solving path constraints without symbolic execution.
Apache License 2.0
916 stars 166 forks source link

Cannot build exiv2-0.26 #38

Open zjuchenyuan opened 5 years ago

zjuchenyuan commented 5 years ago

In your docker environment:

wget http://exiv2.org/releases/exiv2-0.26-trunk.tar.gz
tar zxvf exiv2-0.26-trunk.tar.gz
cd exiv2-trunk

export LLVM_COMPILER=clang
CC=wllvm CXX=wllvm++ CFLAGS=-O0 ./configure --disable-shared
make
cd bin
extract-bc exiv2
# this finished successfully, but the last step fails:
USE_TRACK=1 /angora/bin/angora-clang exiv2.bc -o exiv2.taint
angora-llvm-pass
[+] Track Mode.
ModName: exiv2.bc -- 4171671866
Input is LLVM bitcode
[+] Max constraint id is 113538
#0 0x000000000173c765 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/clang+llvm/bin/clang-4.0+0x173c765)
#1 0x000000000173cdb6 SignalHandler(int) (/clang+llvm/bin/clang-4.0+0x173cdb6)
#2 0x00007f098913d390 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x11390)
#3 0x0000000001170f64 SinkCast(llvm::CastInst*) (/clang+llvm/bin/clang-4.0+0x1170f64)
#4 0x000000000116d040 (anonymous namespace)::CodeGenPrepare::optimizeInst(llvm::Instruction*, bool&) (/clang+llvm/bin/clang-4.0+0x116d040)
#5 0x0000000001168b6e (anonymous namespace)::CodeGenPrepare::runOnFunction(llvm::Function&) (/clang+llvm/bin/clang-4.0+0x1168b6e)
#6 0x00000000013ee183 llvm::FPPassManager::runOnFunction(llvm::Function&) (/clang+llvm/bin/clang-4.0+0x13ee183)
#7 0x00000000013ee373 llvm::FPPassManager::runOnModule(llvm::Module&) (/clang+llvm/bin/clang-4.0+0x13ee373)
#8 0x00000000013ee76a llvm::legacy::PassManagerImpl::run(llvm::Module&) (/clang+llvm/bin/clang-4.0+0x13ee76a)
#9 0x000000000188b4d2 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/clang+llvm/bin/clang-4.0+0x188b4d2)
#10 0x0000000001e4735e clang::CodeGenAction::ExecuteAction() (/clang+llvm/bin/clang-4.0+0x1e4735e)
#11 0x0000000001bad39f clang::FrontendAction::Execute() (/clang+llvm/bin/clang-4.0+0x1bad39f)
#12 0x0000000001b755a8 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/clang+llvm/bin/clang-4.0+0x1b755a8)
#13 0x0000000001c26936 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/clang+llvm/bin/clang-4.0+0x1c26936)
#14 0x0000000000802b0c cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/clang+llvm/bin/clang-4.0+0x802b0c)
#15 0x0000000000801746 main (/clang+llvm/bin/clang-4.0+0x801746)
#16 0x00007f0987eac830 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20830)
#17 0x00000000007febd9 _start (/clang+llvm/bin/clang-4.0+0x7febd9)
Stack dump:
0.      Program arguments: /clang+llvm/bin/clang-4.0 -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name exiv2.bc -mrelocation-model pic -pic-level 1 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -momit-leaf-frame-pointer -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -resource-dir /clang+llvm/bin/../lib/clang/4.0.0 -O3 -fdebug-compilation-dir /d/prog/1exiv2.angora/bin -ferror-limit 19 -fmessage-length 211 -funroll-loops -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -load /angora/bin/unfold-branch-pass.so -load /angora/bin/angora-llvm-pass.so -load /angora/bin/DFSanPass.so -mllvm -TrackMode -mllvm -angora-dfsan-abilist=/angora/bin/angora_abilist.txt -mllvm -angora-dfsan-abilist=/angora/bin/dfsan_abilist.txt -mllvm -angora-exploitation-list=/angora/bin/exploitation_list.txt -mllvm -angora-dfsan-abilist2=/angora/bin/angora_abilist.txt -mllvm -angora-dfsan-abilist2=/angora/bin/dfsan_abilist.txt -o /tmp/exiv2-957ab4.o -x ir exiv2.bc
1.      Code generation
2.      Running pass 'Function Pass Manager' on module 'exiv2.bc'.
3.      Running pass 'CodeGen Prepare' on function '@"dfs$_ZN5Exiv26FileIo8transferERNS_7BasicIoE"'
clang-4.0: error: unable to execute command: Segmentation fault (core dumped)
clang-4.0: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /clang+llvm/bin
clang-4.0: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.

Is this compile error related to sscanf function? Any suggestion for replace sscanf?

zjuchenyuan commented 5 years ago

use angroa-clang instead of wllvm also fails:

CC=/angora/bin/angora-clang CXX=/angora/bin/angora-clang++ LD=/angora/bin/angora-clang ./configure --disable-shared
 USE_TRACK=1 make -j
[+] Track Mode.
ModName: crwimage.cpp -- 1446958078
angora-llvm-pass
[+] Track Mode.
ModName: xmp.cpp -- 2949389276
#0 0x000000000173c765 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/clang+llvm/bin/clang-4.0+0x173c765)
#1 0x000000000173cdb6 SignalHandler(int) (/clang+llvm/bin/clang-4.0+0x173cdb6)
#2 0x00007fc266bbc390 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x11390)
#3 0x0000000001170f64 SinkCast(llvm::CastInst*) (/clang+llvm/bin/clang-4.0+0x1170f64)
#4 0x000000000116d040 (anonymous namespace)::CodeGenPrepare::optimizeInst(llvm::Instruction*, bool&) (/clang+llvm/bin/clang-4.0+0x116d040)
#5 0x0000000001168b6e (anonymous namespace)::CodeGenPrepare::runOnFunction(llvm::Function&) (/clang+llvm/bin/clang-4.0+0x1168b6e)
#6 0x00000000013ee183 llvm::FPPassManager::runOnFunction(llvm::Function&) (/clang+llvm/bin/clang-4.0+0x13ee183)
#7 0x00000000013ee373 llvm::FPPassManager::runOnModule(llvm::Module&) (/clang+llvm/bin/clang-4.0+0x13ee373)
#8 0x00000000013ee76a llvm::legacy::PassManagerImpl::run(llvm::Module&) (/clang+llvm/bin/clang-4.0+0x13ee76a)
#9 0x000000000188b4d2 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/clang+llvm/bin/clang-4.0+0x188b4d2)
#10 0x0000000001e48280 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/clang+llvm/bin/clang-4.0+0x1e48280)
#11 0x000000000222c236 clang::ParseAST(clang::Sema&, bool, bool) (/clang+llvm/bin/clang-4.0+0x222c236)
#12 0x0000000001bad39f clang::FrontendAction::Execute() (/clang+llvm/bin/clang-4.0+0x1bad39f)
#13 0x0000000001b755a8 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/clang+llvm/bin/clang-4.0+0x1b755a8)
#14 0x0000000001c26936 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/clang+llvm/bin/clang-4.0+0x1c26936)
#15 0x0000000000802b0c cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/clang+llvm/bin/clang-4.0+0x802b0c)
#16 0x0000000000801746 main (/clang+llvm/bin/clang-4.0+0x801746)
#17 0x00007fc26592b830 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20830)
#18 0x00000000007febd9 _start (/clang+llvm/bin/clang-4.0+0x7febd9)
Stack dump:
0.      Program arguments: /clang+llvm/bin/clang-4.0 -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name tiffvisitor.cpp -mrelocation-model pic -pic-level 1 -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -momit-leaf-frame-pointer -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -coverage-notes-file /data/exiv2-trunk/src/tiffvisitor.gcno -resource-dir /clang+llvm/bin/../lib/clang/4.0.0 -dependency-file tiffvisitor.d -MT tiffvisitor.o -I ../src -I ../include/ -I ../include/exiv2 -D EXV_LOCALEDIR="/usr/local/share/locale" -I ../xmpsdk/include -D EXV_BUILDING_LIB=1 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/x86_64-linux-gnu/c++/5.4.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/x86_64-linux-gnu/c++/5.4.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/backward -internal-isystem /usr/local/include -internal-isystem /clang+llvm/bin/../lib/clang/4.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -Wall -Wcast-align -Wpointer-arith -Wformat-security -Wmissing-format-attribute -Woverloaded-virtual -W -std=c++98 -fdeprecated-macro -fdebug-compilation-dir /data/exiv2-trunk/src -ferror-limit 19 -fmessage-length 211 -fvisibility hidden -fvisibility-inlines-hidden -funroll-loops -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -load /angora/bin/unfold-branch-pass.so -load /angora/bin/angora-llvm-pass.so -load /angora/bin/DFSanPass.so -mllvm -TrackMode -mllvm -angora-dfsan-abilist=/angora/bin/angora_abilist.txt -mllvm -angora-dfsan-abilist=/angora/bin/dfsan_abilist.txt -mllvm -angora-exploitation-list=/angora/bin/exploitation_list.txt -mllvm -angora-dfsan-abilist2=/angora/bin/angora_abilist.txt -mllvm -angora-dfsan-abilist2=/angora/bin/dfsan_abilist.txt -o tiffvisitor.o -x c++ tiffvisitor.cpp
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'tiffvisitor.cpp'.
4.      Running pass 'CodeGen Prepare' on function '@"dfs$_ZN5Exiv28Internal10TiffReader13readTiffEntryEPNS0_13TiffEntryBaseE"'
clang-4.0: error: unable to execute command: Segmentation fault
clang-4.0: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /clang+llvm/bin
clang-4.0: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
angora-llvm-pass
[+] Track Mode.
ModName: exif.cpp -- 3501100655
clang-4.0: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-4.0: note: diagnostic msg: /tmp/tiffvisitor-9eaf90.cpp
clang-4.0: note: diagnostic msg: /tmp/tiffvisitor-9eaf90.sh
clang-4.0: note: diagnostic msg:

********************
Makefile:195: recipe for target 'tiffvisitor.o' failed
make[1]: *** [tiffvisitor.o] Error 1
spinpx commented 5 years ago

Thanks for your feedback. There are some problems in linking while compiling with libcxx and cxxabi. We still haven't solved these problems.

If you are interested in why it fails. Check how we compile libcxx and cxxabi in https://github.com/AngoraFuzzer/Angora/blob/master/docs/build_target.md#build-c-program-and-c-standard-library. Also, Check how we link these libraries in https://github.com/AngoraFuzzer/Angora/blob/master/llvm_mode/angora-clang.c

spinpx commented 5 years ago

Hi, @zjuchenyuan . I fixed the issue on https://github.com/AngoraFuzzer/Angora/commit/efce5e3deff13701d56e392536a3ec64b47b6312

Please check it. You may need to solve the dependencies of crc and xmlparser. However, can you share me how you build and run the fuzzer with evxiv after you finish it? I will add it to my test dataset for angora. Thanks.

zjuchenyuan commented 5 years ago

ok, build success

The building process is as below. libz and libexpat need to be discarded to avoid ld error. Please fix malformed line error for /angora/tools/gen_library_abilist.sh

wget http://exiv2.org/releases/exiv2-0.26-trunk.tar.gz
tar zxvf exiv2-0.26-trunk.tar.gz
cd exiv2-trunk

export CC=/angora/bin/angora-clang CXX=/angora/bin/angora-clang++ LD=/angora/bin/angora-clang 
./configure --disable-shared
/angora/tools/gen_library_abilist.sh  /usr/lib/x86_64-linux-gnu/libz.so  discard > /tmp/zlib_abilist.txt
/angora/tools/gen_library_abilist.sh  /usr/lib/x86_64-linux-gnu/libexpat.so  discard >> /tmp/zlib_abilist.txt
# and manually edit /tmp/zlib_abilist.txt to remove .so line, otherwise: fatal error: error in backend: error parsing file '/tmp/zlib_abilist.txt': malformed line 1: '/usr/lib/x86_64-linux-gnu/libz.so'

export ANGORA_TAINT_RULE_LIST=/tmp/zlib_abilist.txt
export USE_TRACK=1
make
# now we get bin/exiv2, tainted, about 61MB
# re-run the whole process (exiv2 seems not supporting make clean), unset USE_TRACK to buid fast version, about 27MB