search

DynamoRIO / dynamorio

Dynamic Instrumentation Tool Platform
Other
2.62k stars 556 forks source link

CMake 3.18.0 build of master is broken (Ubuntu 16.04/18.04) #4370

Closed SweetVishnya closed 3 years ago

SweetVishnya commented 4 years ago

commit 12496fe8bb2636fb8df66bca211469b967795414

sudo docker run --rm -it ubuntu:16.04 /bin/bash
apt update && apt upgrade -y && apt install -y git build-essential wget
wget https://github.com/Kitware/CMake/releases/download/v3.18.0/cmake-3.18.0-Linux-x86_64.sh
sh cmake-3.18.0-Linux-x86_64.sh --prefix=/ --exclude-subdir --skip-license
git clone https://github.com/DynamoRIO/dynamorio.git
cd dynamorio
mkdir build
cd build
cmake ..
make

I get the following compilation error.

Scanning dependencies of target dynamorio_static
[  4%] Building C object core/CMakeFiles/dynamorio_static.dir/options.c.o
[  4%] Building C object core/CMakeFiles/dynamorio_static.dir/dynamo.c.o
[  4%] Building C object core/CMakeFiles/dynamorio_static.dir/fragment.c.o
[  4%] Building C object core/CMakeFiles/dynamorio_static.dir/fcache.c.o
[  5%] Building C object core/CMakeFiles/dynamorio_static.dir/link.c.o
[  5%] Building C object core/CMakeFiles/dynamorio_static.dir/dispatch.c.o
[  5%] Building C object core/CMakeFiles/dynamorio_static.dir/emit.c.o
[  5%] Building C object core/CMakeFiles/dynamorio_static.dir/utils.c.o
[  5%] Building C object core/CMakeFiles/dynamorio_static.dir/config.c.o
[  6%] Building C object core/CMakeFiles/dynamorio_static.dir/stats.c.o
[  6%] Building C object core/CMakeFiles/dynamorio_static.dir/heap.c.o
[  6%] Building C object core/CMakeFiles/dynamorio_static.dir/monitor.c.o
[  6%] Building C object core/CMakeFiles/dynamorio_static.dir/perfctr.c.o
[  7%] Building C object core/CMakeFiles/dynamorio_static.dir/vmareas.c.o
[  7%] Building C object core/CMakeFiles/dynamorio_static.dir/rct.c.o
[  7%] Building C object core/CMakeFiles/dynamorio_static.dir/hotpatch.c.o
[  7%] Building C object core/CMakeFiles/dynamorio_static.dir/hashtable.c.o
[  7%] Building C object core/CMakeFiles/dynamorio_static.dir/module_list.c.o
[  8%] Building C object core/CMakeFiles/dynamorio_static.dir/moduledb.c.o
[  8%] Building C object core/CMakeFiles/dynamorio_static.dir/perscache.c.o
[  8%] Building C object core/CMakeFiles/dynamorio_static.dir/nudge.c.o
[  8%] Building C object core/CMakeFiles/dynamorio_static.dir/synch.c.o
[  8%] Building C object core/CMakeFiles/dynamorio_static.dir/buildmark.c.o
[  9%] Building C object core/CMakeFiles/dynamorio_static.dir/loader_shared.c.o
[  9%] Building C object core/CMakeFiles/dynamorio_static.dir/io.c.o
[  9%] Building C object core/CMakeFiles/dynamorio_static.dir/native_exec.c.o
[  9%] Building C object core/CMakeFiles/dynamorio_static.dir/lib/instrument.c.o
[ 10%] Building C object core/CMakeFiles/dynamorio_static.dir/translate.c.o
[ 10%] Building C object core/CMakeFiles/dynamorio_static.dir/annotations.c.o
[ 10%] Building C object core/CMakeFiles/dynamorio_static.dir/jit_opt.c.o
[ 10%] Building C object core/CMakeFiles/dynamorio_static.dir/string.c.o
[ 10%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/arch.c.o
[ 11%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/emit_utils_shared.c.o
[ 11%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/x86/emit_utils.c.o
[ 11%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/opnd_shared.c.o
[ 11%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/opnd.c.o
[ 12%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/instr_shared.c.o
[ 12%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/instr.c.o
[ 12%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/instrlist.c.o
[ 12%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/decode_shared.c.o
[ 12%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/decode.c.o
[ 13%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/encode_shared.c.o
[ 13%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/encode.c.o
[ 13%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/disassemble_shared.c.o
[ 13%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/disassemble.c.o
[ 13%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/ir_utils_shared.c.o
[ 14%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/ir_utils.c.o
[ 14%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/decode_table.c.o
[ 14%] Building C object core/CMakeFiles/dynamorio_static.dir/ir/x86/decode_fast.c.o
[ 14%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/interp.c.o
[ 15%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/proc_shared.c.o
[ 15%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/x86/proc.c.o
[ 15%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/mangle_shared.c.o
[ 15%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/x86/mangle.c.o
[ 15%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/clean_call_opt_shared.c.o
[ 16%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/x86/clean_call_opt.c.o
[ 16%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/steal_reg.c.o
[ 16%] Building C object core/CMakeFiles/dynamorio_static.dir/arch/x86_code.c.o
[ 16%] Building ASM object core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o
cpp: error: STATIC_LIBRARY: No such file or directory
cpp: warning: '-x c' after last input file has no effect
cpp: error: unrecognized command line option '--defsym'
cpp: fatal error: no input files
compilation terminated.
core/CMakeFiles/dynamorio_static.dir/build.make:838: recipe for target 'core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o' failed
make[2]: *** [core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o] Error 1
CMakeFiles/Makefile2:751: recipe for target 'core/CMakeFiles/dynamorio_static.dir/all' failed
make[1]: *** [core/CMakeFiles/dynamorio_static.dir/all] Error 2
Makefile:170: recipe for target 'all' failed
make: *** [all] Error 2
VERBOSE=1 make
[  4%] Building ASM object core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o
cd /dynamorio/build/core && /usr/bin/cpp  -O2 -g -DNDEBUG -fPIC -I/dynamorio/core/drlibc -I/dynamorio/core/ir/x86 -I/dynamorio/core/arch/x86 -I/dynamorio/core/unix -I/dynamorio/core/ir -I/dynamorio/core/arch -I/dynamorio/core/lib -I/dynamorio/build -I/dynamorio/build/include/annotations --defsym STATIC_LIBRARY -DCPP2ASM -E /dynamorio/core/arch/x86/x86.asm -o CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o.s
cpp: error: STATIC_LIBRARY: No such file or directory
cpp: warning: '-x c' after last input file has no effect
cpp: error: unrecognized command line option '--defsym'
cpp: fatal error: no input files
compilation terminated.
core/CMakeFiles/dynamorio_static.dir/build.make:838: recipe for target 'core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o' failed
make[2]: *** [core/CMakeFiles/dynamorio_static.dir/arch/x86/x86.asm.o] Error 1
make[2]: Leaving directory '/dynamorio/build'
CMakeFiles/Makefile2:751: recipe for target 'core/CMakeFiles/dynamorio_static.dir/all' failed
make[1]: *** [core/CMakeFiles/dynamorio_static.dir/all] Error 2
make[1]: Leaving directory '/dynamorio/build'
Makefile:170: recipe for target 'all' failed
make: *** [all] Error 2
SweetVishnya commented 4 years ago

I am also experiencing it on my working Ubuntu 18.04.4. But I was not able to reproduce it in docker.

UPD: I was wrong. I am able to reproduce it on Ubuntu 18.04. Just replace the first line with:

sudo docker run --rm -it ubuntu:18.04 /bin/bash
SweetVishnya commented 4 years ago

Maybe, it is somehow connected with the newest Ubuntu updates.

SweetVishnya commented 4 years ago

It seems to be a problem with a new CMake. With older one DynamoRIO builds fine:

wget https://github.com/Kitware/CMake/releases/download/v3.17.3/cmake-3.17.3-Linux-x86_64.sh
higsyuhing commented 4 years ago

modify file: ./build/core/CMakeFiles/dynamorio.dir/flags.make at line 6, change to: ASM_DEFINES = --def=dynamorio_EXPORTS

This file is dynamically modified, you need to update it after you meet the compilation error. [for maintainer] Changing configure file? can solve this?

But I still get linking error after eliminating 1. "-Werror" for signal.h, and 2. update "--defsys", when: Linking C shared library ../lib64/release/libdynamorio.so

woodruffw commented 3 years ago

I've also run into this, running on Ubuntu 18.04: https://github.com/trailofbits/mishegos/pull/252

woodruffw commented 3 years ago

(And for more context, all of GitHub Actions has recently switched to CMake 3.19 for ubuntu-latest: https://github.com/actions/virtual-environments/pull/2159)

woodruffw commented 3 years ago

I did a quick search, and my educated guess is that this CMake helper file is probably the source of the problem: https://github.com/DynamoRIO/dynamorio/blob/master/make/cpp2asm_support.cmake. I'll keep looking into this.

In particular, this is suspect: https://github.com/DynamoRIO/dynamorio/blob/536cbd0a2f895294b9efa08d5bc35129f9bc2b8b/make/cpp2asm_support.cmake#L326-L341

It's calling the preprocessor first, so that's probably where the flag confusion is happening.

derekbruening commented 3 years ago

Our CI is still on Ubuntu 16.04 (could upgrade...but nobody wants to deal w/ the probably flaky tests that will flare up on any CI change...)

But, I now have cmake 3.18.4 locally and I do hit a problem:

$ cmake --version
cmake version 3.18.4
$ cmake -GNinja -DDEBUG=ON -DBUILD_TESTS=ON ../src && ninja
...
[103/1416] Building ASM object core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o
FAILED: core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o 
/usr/bin/cpp  -g --MD core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o.d -I/home/bruening/dr/git/src/core/drlibc -I/home/bruening/dr/git/src/core/ir/x86 -I/home/bruening/dr/git/src/core/arch/x86 -I/home/bruening/dr/git/src/core/unix -I/home/bruening/dr/git/src/core/ir -I/home/bruening/dr/git/src/core/arch -I/home/bruening/dr/git/src/core/lib -I. -Iinclude/annotations  -DCPP2ASM -E /home/bruening/dr/git/src/core/arch/x86/memfuncs.asm -o core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o.s && /usr/bin/cmake -Dfile=core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o.s -P "/home/bruening/dr/git/src/make/CMake_asm.cmake" && /usr/bin/as -mmnemonic=intel -msyntax=intel -mnaked-reg --64 --noexecstack -g -o core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o.s
cpp: error: core/CMakeFiles/drmemfuncs.dir/arch/x86/memfuncs.asm.o.d: No such file or directory
cpp: error: unrecognized command-line option ‘--MD’; did you mean ‘-MD’?
cpp: fatal error: no input files
compilation terminated.
derekbruening commented 3 years ago

It looks like Ubuntu 16.04 on Github Actions suddenly upgraded CMake and is hitting this: https://github.com/DynamoRIO/dynamorio/pull/4580/checks?check_run_id=1494251904

debug-internal-64: **** 19 build errors ****
    cpp: error: STATIC_LIBRARY: No such file or directory
    cpp: error: unrecognized command line option ‘--defsym’
    core/CMakeFiles/dynamorio_static_nohide.dir/build.make:838: recipe for target 'core/CMakeFiles/dynamorio_static_nohide.dir/arch/x86/x86.asm.o' failed
    make[2]: *** [core/CMakeFiles/dynamorio_static_nohide.dir/arch/x86/x86.asm.o] Error 1
    make[1]: *** [core/CMakeFiles/dynamorio_static_nohide.dir/all] Error 2
    CMakeFiles/Makefile2:2302: recipe for target 'core/CMakeFiles/dynamorio_static_nohide.dir/all' failed
    cpp: error: dynamorio_EXPORTS: No such file or directory
    cpp: error: unrecognized command line option ‘--defsym’
    core/CMakeFiles/dynamorio.dir/build.make:838: recipe for target 'core/CMakeFiles/dynamorio.dir/arch/x86/x86.asm.o' failed
    make[2]: *** [core/CMakeFiles/dynamorio.dir/arch/x86/x86.asm.o] Error 1
derekbruening commented 3 years ago

They're on 3.19 and it also complains here:

2020-12-03T13:55:57.4900065Z CMake Deprecation Warning at /home/runner/work/dynamorio/dynamorio/suite/runsuite.cmake:32 (cmake_minimum_required):
2020-12-03T13:55:57.4901325Z   Compatibility with CMake < 2.8.12 will be removed from a future version of
2020-12-03T13:55:57.4902031Z   CMake.
2020-12-03T13:55:57.4902333Z 
2020-12-03T13:55:57.4902845Z   Update the VERSION argument <min> value or use a ...<max> suffix to tell
2020-12-03T13:55:57.4903575Z   CMake that the project does not need compatibility with older versions.
derekbruening commented 3 years ago

@abhinav92003 pointed to the GA failures coming from this recent update: https://github.com/actions/virtual-environments/pull/2159

abhinav92003 commented 3 years ago

@abhinav92003 pointed to the GA failures coming from this recent update: actions/virtual-environments#2159

Actually I came to know of it from woodruffw's comment above :)

derekbruening commented 3 years ago

Locally, if I solve the --MD issue, I then repro the --defsym error. After solving that I then hit this:

/usr/bin/ld: core/CMakeFiles/unit_tests.dir/arch/x86/x86.asm.o: in function `_start':
/home/bruening/dr/git/src/core/arch/x86/x86.asm:1158: multiple definition of `_start'; /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/Scrt1.o:(.text+0x0): first defined here
/usr/bin/ld: core/CMakeFiles/unit_tests.dir/arch/x86/x86.asm.o: in function `_start':
/home/bruening/dr/git/src/core/arch/x86/x86.asm:1165: undefined reference to `relocate_dynamorio'
/usr/bin/ld: core/CMakeFiles/unit_tests.dir/arch/x86/x86.asm.o: in function `reloaded_xfer':
/home/bruening/dr/git/src/core/arch/x86/x86.asm:1179: undefined reference to `privload_early_inject'
collect2: error: ld returned 1 exit status
derekbruening commented 3 years ago

So we have two issues:

1) The CMake-generated define -Ddynamorio_EXPORTS is being passed as --defsym dynamorio_EXPORTS in <DEFINES> for asm targets, but we pass that to cpp.

2) Various compiler flags like --MD are in <FLAGS> and cpp does not like them.

My solution is to: 1) set(CMAKE_ASM_DEFINE_FLAG "-D") 2) Remove <FLAGS> from the cpp command for asm. 3) Move all of our preprocessor defines from COMPILE_FLAGS properties with explicit -D to COMPILE_DEFINITIONS properites without -D so we get them in <DEFINES>.

derekbruening commented 3 years ago

I have that big switch from COMPILE_FLAGS to COMPILE_DEFINITIONS working for everything except the avx tests which want the built-in compiler defs like __AVX__ in the asm code: and it's not there now. Trying to solve this last bit.