search

lifting-bits / remill

Library for lifting machine code to LLVM bitcode
Apache License 2.0
1.27k stars 143 forks source link

make test_dependencies fail #468

Closed adahsuzixin closed 3 years ago

adahsuzixin commented 3 years ago

After building Remill with clang, Run the test suite.

 cd remill/scripts/remill-build/
 make test_dependencies

make: *** No rule to make target 'test_dependencies'. Stop.

ekilmer commented 3 years ago

Hmmmm. Usually this only happens when not using clang to build remill. Could you please post the output from cmake when you run its initial configuration (the first cmake command run before building).

adahsuzixin commented 3 years ago 
[-] Library version is libraries-llvm-10-ubuntu-18.04-amd64
Fetching: https://github.com/trailofbits/cxx-common/releases/latest/download/libraries-llvm-10-ubuntu-18.04-amd64.tar.xz
-- The C compiler identification is GNU 8.3.1
-- The CXX compiler identification is GNU 8.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Install prefix: /usr/local
-- The following compiler has been selected to compile the bitcode: /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found Z3: /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/z3/lib/libz3.a (found suitable version "4.8.8", minimum required is "4.7.1")
-- ccache: enabled with '/usr/local/bin/ccache'. The cache folder is located here: '/root/.ccache'
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/c++
-- Configuring done
-- Generating done
-- Build files have been written to: /home/suzixin/remill_opensource/remill/scripts/remill-build
adahsuzixin commented 3 years ago

Yeah, This is the root cause.

       -DCMAKE_BC_COMPILER="${TRAILOFBITS_LIBRARIES}/llvm/bin/clang++" \
       -DCMAKE_BC_LINKER="${TRAILOFBITS_LIBRARIES}/llvm/bin/llvm-link" \
       -DCMAKE_VERBOSE_MAKEFILE=True \
+      -DCMAKE_C_COMPILER="${TRAILOFBITS_LIBRARIES}/llvm/bin/clang" \
+      -DCMAKE_CXX_COMPILER="${TRAILOFBITS_LIBRARIES}/llvm/bin/clang++" \
       ${BUILD_FLAGS} \
       "${SRC_DIR}"

After modification, things works. However, I got another question when compiling.

# make test_dependencies
-- Install prefix: /usr/local
-- The following compiler has been selected to compile the bitcode: /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++
-- ccache: enabled with '/usr/local/bin/ccache'. The cache folder is located here: '/root/.ccache'
 > Generating runtime target: aarch64
 > Generating runtime target: sparc32
 > Generating runtime target: sparc64
 > Generating runtime target: x86
 > Generating runtime target: x86_avx
 > Generating runtime target: x86_avx512
 > Generating runtime target: amd64
 > Generating runtime target: amd64_avx
 > Generating runtime target: amd64_avx512
-- X86 tests enabled
-- Adding test: x86 as run-x86-tests
-- Adding test: x86_avx as run-x86_avx-tests
-- Adding test: amd64 as run-amd64-tests
-- Adding test: amd64_avx as run-amd64_avx-tests
-- Configuring done
-- Generating done
-- Build files have been written to: /home/suzixin/remill_opensource/remill/scripts/remill-build
[  1%] Checking the git repository for changes...
[  1%] Built target check_git_remill
[  4%] Built target remill_version
[  7%] Built target remill_arch_x86
[ 12%] Built target remill_arch_aarch64
[ 17%] Built target remill_arch_sparc32
[ 22%] Built target remill_arch_sparc64
[ 28%] Built target remill_arch
[ 41%] Built target remill_bc
[ 47%] Built target remill_os
Scanning dependencies of target lift-x86_avx-tests
[ 49%] Building CXX object tests/X86/CMakeFiles/lift-x86_avx-tests.dir/Lift.cpp.o
/home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++: /lib64/libtinfo.so.5: no version information available (required by /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++)
[ 50%] Building ASM object tests/X86/CMakeFiles/lift-x86_avx-tests.dir/Tests.S.o
/home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++: /lib64/libtinfo.so.5: no version information available (required by /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++)
/home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang-10: /lib64/libtinfo.so.5: no version information available (required by /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang-10)
/home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang-10: /lib64/libtinfo.so.5: no version information available (required by /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang-10)
[ 52%] Linking CXX executable lift-x86_avx-tests
/home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++: /lib64/libtinfo.so.5: no version information available (required by /home/suzixin/remill_opensource/remill/scripts/remill-build/libraries/llvm/bin/clang++)
[ 52%] Built target lift-x86_avx-tests
[ 52%] Built target semantics
[ 53%] Generating tests_x86_avx.bc
F20201229 09:26:24.287413 637461 InstructionLifter.cpp:576] Check failed: val_type->isIntegerTy() Expected XMM0 to be an integral type for instruction at 559ea0
*** Check failure stack trace: ***
    @           0xd968dc  google::LogMessageFatal::~LogMessageFatal()
    @           0x55e8eb  remill::InstructionLifter::LiftRegisterOperand()
    @           0x55c2a4  remill::InstructionLifter::LiftIntoBlock()
    @           0x562539  remill::TraceLifter::Impl::Lift()
    @           0x561c8c  remill::TraceLifter::Lift()
    @           0x557e7b  main
    @     0x7f2a78af7873  __libc_start_main
    @           0x55798e  _start
make[3]: *** [tests/X86/CMakeFiles/run-x86_avx-tests.dir/build.make:85: tests/X86/tests_x86_avx.bc] Aborted (core dumped)
make[2]: *** [CMakeFiles/Makefile2:2088: tests/X86/CMakeFiles/run-x86_avx-tests.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:463: CMakeFiles/test_dependencies.dir/rule] Error 2
make: *** [Makefile:194: test_dependencies] Error 2
adahsuzixin commented 3 years ago

I found the problem when lifting the "CVTSI2SS_XMMss_GPR32d" inst form. After comment this test file, problem solved.

diff --git a/tests/X86/Tests.S b/tests/X86/Tests.S
index 7126f84..cd86f3a 100644
--- a/tests/X86/Tests.S
+++ b/tests/X86/Tests.S
@@ -434,8 +434,8 @@ SYMBOL(__x86_test_table_begin):
 #include "tests/X86/CONVERT/CVTPS2PD.S"
 #include "tests/X86/CONVERT/CVTSD2SI.S"
 #include "tests/X86/CONVERT/CVTSD2SS.S"
-#include "tests/X86/CONVERT/CVTSI2SD.S"
-#include "tests/X86/CONVERT/CVTSI2SS.S"
+// #include "tests/X86/CONVERT/CVTSI2SD.S"
+// #include "tests/X86/CONVERT/CVTSI2SS.S"
 #include "tests/X86/CONVERT/CVTSS2SD.S"
 #include "tests/X86/CONVERT/CVTSS2SI.S"
 #include "tests/X86/CONVERT/CVTTPD2DQ.S"
pgoodman commented 3 years ago

The problem is that there is a difference between the types in the DEF_DEM and what is being produced by the decoder. The fix will need to update the semantics slightly.

artemdinaburg commented 3 years ago

Closing since make test_dependencies is working for me on multiple checkouts and architectures