llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.59k stars 11.82k forks source link

[runtimes] [cmake] [19.1.1] Possible bug with usage of `CLANG_DEFAULT_UNWINDLIB=libunwind` and runtime builds #111667

Open Crivella opened 1 week ago

Crivella commented 1 week ago

I am performing a multi-stage build of the LLVM projects 19.1.1.

It seems that during the 2nd/3rd stage of the build when clang is build with CLANG_DEFAULT_UNWINDLIB=libunwind it fails to detect the ABI info of the C compiler during the re-run of cmake for the runtimes. This seems to be caused by the fact that clang automatically links with -lunwind when it is still not available in the respeective ../../lib/<triple> dir.

Log of the CMake call from `runtimes/runtimes-bins/CMakeFiles/CMakeConfigureLog.yaml` ```yaml - kind: "try_compile-v1" backtrace: - "/home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/share/cmake-3.29/Modules/CMakeDetermineCompilerABI.cmake:67 (try_compile)" - "/home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/share/cmake-3.29/Modules/CMakeTestCCompiler.cmake:26 (CMAKE_DETERMINE_COMPILER_ABI)" - "CMakeLists.txt:11 (project)" checks: - "Detecting C compiler ABI info" directories: source: "/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ" binary: "/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ" cmakeVariables: CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS: "/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin/clang-scan-deps" CMAKE_CXX_COMPILER_TARGET: "x86_64-unknown-linux-gnu" CMAKE_C_COMPILER_TARGET: "x86_64-unknown-linux-gnu" CMAKE_C_FLAGS: "--gcc-install-dir=/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0" CMAKE_C_FLAGS_DEBUG: "-g" CMAKE_EXE_LINKER_FLAGS: "" buildResult: variable: "CMAKE_C_ABI_COMPILED" cached: true stdout: | Change Dir: '/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ' Run Build Command(s): /home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile cmTC_fcd96/fast gmake[3]: Entering directory '/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ' /usr/bin/gmake -f CMakeFiles/cmTC_fcd96.dir/build.make CMakeFiles/cmTC_fcd96.dir/build gmake[4]: Entering directory '/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ' Building C object CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin/clang --target=x86_64-unknown-linux-gnu --gcc-install-dir=/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0 -v -MD -MT CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o -MF CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o.d -o CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o -c /home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/share/cmake-3.29/Modules/CMakeCCompilerABI.c clang version 19.1.1 Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin Selected GCC installation: /home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0 Candidate multilib: .;@m64 Selected multilib: .;@m64 (in-process) "/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin/clang-19" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name CMakeCCompilerABI.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ -v -fcoverage-compilation-dir=/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ -resource-dir /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19 -dependency-file CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o.d -MT CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o -sys-header-deps -I/home/crivella/.local/easybuild/software/zlib/1.3.1/include -I/home/crivella/.local/easybuild/software/ncurses/6.5/include -I/home/crivella/.local/easybuild/software/gettext/0.22.5-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/libiconv/1.17-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/libxml2/2.12.7-GCCcore-13.3.0/include/libxml2 -I/home/crivella/.local/easybuild/software/libxml2/2.12.7-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/expat/2.6.2-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/Python/3.12.3-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/libffi/3.4.5-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/SQLite/3.45.3-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/Tcl/8.6.14-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/libreadline/8.2-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/libarchive/3.7.4-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/XZ/5.4.5-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/cURL/8.7.1-GCCcore-13.3.0/include -I/home/crivella/.local/easybuild/software/OpenSSL/3/include -I/home/crivella/.local/easybuild/software/bzip2/1.0.8-GCCcore-13.3.0/include -internal-isystem /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/include -internal-isystem /usr/local/include -internal-isystem /home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o -x c /home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/share/cmake-3.29/Modules/CMakeCCompilerABI.c clang -cc1 version 19.1.1 based upon LLVM 19.1.1 default target x86_64-unknown-linux-gnu ignoring nonexistent directory "/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0/../../../../x86_64-pc-linux-gnu/include" ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /home/crivella/.local/easybuild/software/zlib/1.3.1/include /home/crivella/.local/easybuild/software/ncurses/6.5/include /home/crivella/.local/easybuild/software/gettext/0.22.5-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/libiconv/1.17-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/libxml2/2.12.7-GCCcore-13.3.0/include/libxml2 /home/crivella/.local/easybuild/software/libxml2/2.12.7-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/expat/2.6.2-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/Python/3.12.3-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/libffi/3.4.5-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/SQLite/3.45.3-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/Tcl/8.6.14-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/libreadline/8.2-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/libarchive/3.7.4-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/XZ/5.4.5-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/cURL/8.7.1-GCCcore-13.3.0/include /home/crivella/.local/easybuild/software/OpenSSL/3/include /home/crivella/.local/easybuild/software/bzip2/1.0.8-GCCcore-13.3.0/include /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/include /usr/local/include /usr/include/x86_64-linux-gnu /usr/include End of search list. Linking C executable cmTC_fcd96 /home/crivella/.local/easybuild/software/CMake/3.29.3-GCCcore-13.3.0/bin/cmake -E cmake_link_script CMakeFiles/cmTC_fcd96.dir/link.txt --verbose=1 /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin/clang --target=x86_64-unknown-linux-gnu --gcc-install-dir=/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0 -v -Wl,-v CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o -o cmTC_fcd96 clang version 19.1.1 Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin Selected GCC installation: /home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0 Candidate multilib: .;@m64 Selected multilib: .;@m64 "/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/bin/ld.lld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o cmTC_fcd96 /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/lib/x86_64-unknown-linux-gnu/clang_rt.crtbegin.o -L/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/lib/x86_64-unknown-linux-gnu -L/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0 -L/home/crivella/.local/easybuild/software/GCCcore/13.3.0/lib/gcc/x86_64-pc-linux-gnu/13.3.0/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib -L/home/crivella/.local/easybuild/software/zlib/1.3.1/lib -L/home/crivella/.local/easybuild/software/ncurses/6.5/lib -L/home/crivella/.local/easybuild/software/Perl/5.38.2-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/gettext/0.22.5-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/libiconv/1.17-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/libxml2/2.12.7-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/expat/2.6.2-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/lit/18.1.8-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/Python/3.12.3-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/psutil/6.0.0-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/libffi/3.4.5-GCCcore-13.3.0/lib64 -L/home/crivella/.local/easybuild/software/libffi/3.4.5-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/SQLite/3.45.3-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/Tcl/8.6.14-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/libreadline/8.2-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/binutils/2.42-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/libarchive/3.7.4-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/XZ/5.4.5-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/cURL/8.7.1-GCCcore-13.3.0/lib -L/home/crivella/.local/easybuild/software/OpenSSL/3/lib -L/home/crivella/.local/easybuild/software/bzip2/1.0.8-GCCcore-13.3.0/lib -v CMakeFiles/cmTC_fcd96.dir/CMakeCCompilerABI.c.o /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a --as-needed -lunwind --no-as-needed -lc /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a --as-needed -lunwind --no-as-needed /home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/lib/clang/19/lib/x86_64-unknown-linux-gnu/clang_rt.crtend.o /lib/x86_64-linux-gnu/crtn.o LLD 19.1.1 (compatible with GNU linkers) ld.lld: error: unable to find library -lunwind ld.lld: error: unable to find library -lunwind clang: error: linker command failed with exit code 1 (use -v to see invocation) gmake[4]: *** [CMakeFiles/cmTC_fcd96.dir/build.make:100: cmTC_fcd96] Error 1 gmake[4]: Leaving directory '/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ' gmake[3]: *** [Makefile:127: cmTC_fcd96/fast] Error 2 gmake[3]: Leaving directory '/home/crivella/.local/easybuild/build/LLVM/19.1.1/system-system/llvm.obj.3/runtimes/runtimes-bins/CMakeFiles/CMakeScratch/TryCompile-xJcaVZ' exitCode: 2 ```

I think this was not a problem with LLVM < 19 (eg 18.1.8) as the compilation still ran successfully and passed the test-suite with a few other unrelated errors. The problem I am having with 19.1.1 is the check at https://github.com/llvm/llvm-project/blob/a9f5a44aa0032c6efb262d2d5f79847045e525f1/offload/CMakeLists.txt#L21, with CMAKE_SIZEOF_VOID_P not being set causing the skip of the offload project.

I tried adding to CMAKE_EXE_LINKER_FLAGS a -L to the previous stage lib/<triple> which makes the ABI detection successful and does not skip the offload project, but it than causes others problem down the line

Crivella commented 1 week ago

Upon further inspection, to also try and understand why the runtimes afterward are not failing the linking, i scoured a bit the logs and the CMake files.

I think this is related to https://github.com/llvm/llvm-project/blob/a9f5a44aa0032c6efb262d2d5f79847045e525f1/runtimes/CMakeLists.txt#L107

In my case i was able to solve this but adding --unwindlib=none to CMAKE_EXE_LINKER_FLAGS (through RUNTIMES_CMAKE_ARGS), not sure if there is a way to do it at the CMake level before the invocation of project