search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
27.84k stars 11.46k forks source link

LLVM_ENABLE_RUNTIMES broken on linux/musl #49139

Open arachsys opened 3 years ago

arachsys commented 3 years ago
Bugzilla Link 49795
Version trunk
OS Linux
CC @arachsys

Extended Description

TL;DR: LLVM_ENABLE_RUNTIMES in both release/12.x and main appears to be broken on linux/musl but not on linux/glibc.

I stumbled over this whilst packaging llvm/clang/etc for a weird local linux distribution, but I reproduce and demonstrate below with mainstream Alpine Linux and Debian images.

In a linux/musl image, such as

https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-minirootfs-3.13.3-x86_64.tar.gz
apk add alpine-sdk cmake linux-headers python3

running

mkdir build && cd build
cmake -W no-dev \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang;compiler-rt;libunwind;libcxx;libcxxabi" \
  -DLLVM_INCLUDE_TESTS=No \
  -DLLVM_TARGETS_TO_BUILD="Native" \
  -DLIBCXX_HAS_MUSL_LIBC=Yes \
  -DCOMPILER_RT_BUILD_LIBFUZZER=No \
  -DCOMPILER_RT_BUILD_MEMPROF=No \
  -DCOMPILER_RT_BUILD_SANITIZERS=No \
  -DCOMPILER_RT_BUILD_XRAY=No \
  ../llvm && make -j16

completes successfully, and the resulting compiler and runtimes can be re-used to fully bootstrap. However, the same thing with ENABLE_RUNTIMES:

cmake -W no-dev \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang" \
  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libunwind;libcxx;libcxxabi" \
  -DLLVM_INCLUDE_TESTS=No \
  -DLLVM_TARGETS_TO_BUILD="Native" \
  -DLIBCXX_HAS_MUSL_LIBC=Yes \
  -DCOMPILER_RT_BUILD_LIBFUZZER=No \
  -DCOMPILER_RT_BUILD_MEMPROF=No \
  -DCOMPILER_RT_BUILD_SANITIZERS=No \
  -DCOMPILER_RT_BUILD_XRAY=No \
  ../llvm && make -j16

crashes when the runtimes build tries to configure:

[...]
-- Performing Test COMPILER_RT_HAS_FUSE_LD_LLD_FLAG
-- Performing Test COMPILER_RT_HAS_FUSE_LD_LLD_FLAG - Failed
-- Performing Test COMPILER_RT_HAS_GNU_VERSION_SCRIPT_COMPAT
-- Performing Test COMPILER_RT_HAS_GNU_VERSION_SCRIPT_COMPAT - Failed
-- Performing Test COMPILER_RT_HAS_VERSION_SCRIPT
-- Performing Test COMPILER_RT_HAS_VERSION_SCRIPT - Failed
CMake Error at /root/llvm-project/compiler-rt/cmake/config-ix.cmake:196 (message):
  Please use architecture with 4 or 8 byte pointers.
Call Stack (most recent call first):
  /root/llvm-project/compiler-rt/CMakeLists.txt:245 (include)

-- Configuring incomplete, errors occurred!
See also "/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeOutput.log".
See also "/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeError.log".
make[2]: *** [runtimes/CMakeFiles/runtimes.dir/build.make:126: runtimes/runtimes-stamps/runtimes-configure] Error 1
make[1]: *** [CMakeFiles/Makefile2:34100: runtimes/CMakeFiles/runtimes.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

During this configure step, every runtimes-bin test is down as 'Failed', even things like 'C_SUPPORTS_FPIC' or 'Looking for fopen in c'. Looking at runtimes/runtimes-bin/CMakeFiles/CMakeError.log shows link repeatedly failing like this:

Run Build Command(s):/usr/bin/make cmTC_ac0dd/fast && make[3]: Entering directory '/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp'
/usr/bin/make  -f CMakeFiles/cmTC_ac0dd.dir/build.make CMakeFiles/cmTC_ac0dd.dir/build
make[4]: Entering directory '/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_ac0dd.dir/src.cxx.o
/root/llvm-project/build2/./bin/clang++ --target=x86_64-unknown-linux-gnu   -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-comment -DCOMPILER_RT_HAS_VERSION_SCRIPT  -nostdinc++  -isystem /root/llvm-project/runtimes/../libcxx/include -Wl,--version-script,/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp/dummy.vers -o CMakeFiles/cmTC_ac0dd.dir/src.cxx.o -c /root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp/src.cxx
clang-12: warning: -Wl,--version-script,/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp/dummy.vers: 'linker' input unused [-Wunused-command-line-argument]
Linking CXX executable cmTC_ac0dd
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_ac0dd.dir/link.txt --verbose=1
/root/llvm-project/build2/./bin/clang++ --target=x86_64-unknown-linux-gnu  -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-comment -DCOMPILER_RT_HAS_VERSION_SCRIPT  -nostdinc++  -isystem /root/llvm-project/runtimes/../libcxx/include -Wl,--version-script,/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp/dummy.vers CMakeFiles/cmTC_ac0dd.dir/src.cxx.o -o cmTC_ac0dd
clang-12: warning: argument unused during compilation: '-nostdinc++' [-Wunused-command-line-argument]
/usr/bin/ld: cannot find crtbegin.o: No such file or directory
/usr/bin/ld: cannot find -lgcc
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [CMakeFiles/cmTC_ac0dd.dir/build.make:106: cmTC_ac0dd] Error 1
make[4]: Leaving directory '/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp'
make[3]: *** [Makefile:140: cmTC_ac0dd/fast] Error 2
make[3]: Leaving directory '/root/llvm-project/build2/runtimes/runtimes-bins/CMakeFiles/CMakeTmp'

This doesn't happen on a linux/glibc image, such as Debian unstable with

https://raw.githubusercontent.com/debuerreotype/docker-debian-artifacts/dist-amd64/unstable/rootfs.tar.xz
apt-get install build-essential cmake python3 python3-distutils

where

  cmake -W no-dev \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang" \
  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libunwind;libcxx;libcxxabi" \
  -DLLVM_INCLUDE_TESTS=No \
  -DLLVM_TARGETS_TO_BUILD="Native" \
  -DCOMPILER_RT_BUILD_LIBFUZZER=No \
  -DCOMPILER_RT_BUILD_MEMPROF=No \
  -DCOMPILER_RT_BUILD_SANITIZERS=No \
  -DCOMPILER_RT_BUILD_XRAY=No \
  ../llvm && make -j16

completes fine, as does a build with the runtimes in LLVM_ENABLE_PROJECTS. (Same cmake command except I've dropped -DLIBCXX_HAS_MUSL_LIBC.)

arachsys commented 3 years ago

Following up to this: the problem seems to be that the musl builds are detecting their host triple as x86_64-unknown-linux-gnu instead of x86_64-unknown-linux-musl or x86_64-alpine-linux-musl. (See 'build/bin/clang -dumpmachine'.)

Adding an explicit

-DLLVM_HOST_TRIPLE=x86_64-alpine-linux-musl

is sufficient to get the ENABLE_RUNTIMES version to build.

Endilll commented 12 months ago

@arachsys Are you still experiencing this issue on trunk?

arachsys commented 11 months ago

Hi, I've just re-checked this. Constructing an up-to-date test build chroot from completely stock Alpine Linux:

https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.3-x86_64.tar.gz
apk add alpine-sdk cmake linux-headers python3

and compiling the HEAD of git main as of now, c336a061, the following recipe still fails:

cmake -W no-dev \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang" \
  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libunwind;libcxx;libcxxabi" \
  -DLLVM_INCLUDE_TESTS=No \
  -DLLVM_TARGETS_TO_BUILD="Native" \
  -DLIBCXX_HAS_MUSL_LIBC=Yes \
  -DCOMPILER_RT_BUILD_LIBFUZZER=No \
  -DCOMPILER_RT_BUILD_MEMPROF=No \
  -DCOMPILER_RT_BUILD_SANITIZERS=No \
  -DCOMPILER_RT_BUILD_XRAY=No \
  ../llvm && make -j16

whereas adding -DLLVM_HOST_TRIPLE=x86_64-alpine-linux-musl gets it building fine.

Nowadays the failure is a little more obvious to diagnose, because there's an explicit cmake warning about not knowing the target architecture:

-- Performing Test CXX_SUPPORTS_UNWINDLIB_EQ_NONE_FLAG
-- Performing Test CXX_SUPPORTS_UNWINDLIB_EQ_NONE_FLAG - Failed
-- Performing Test CXX_SUPPORTS_NOSTDLIBXX_FLAG
-- Performing Test CXX_SUPPORTS_NOSTDLIBXX_FLAG - Failed
-- Performing Test CXX_SUPPORTS_NOSTDINCXX_FLAG
-- Performing Test CXX_SUPPORTS_NOSTDINCXX_FLAG - Failed
CMake Warning (dev) at /usr/share/cmake/Modules/GNUInstallDirs.cmake:243 (message):
  Unable to determine default CMAKE_INSTALL_LIBDIR directory because no
  target architecture is known.  Please enable at least one language before
  including GNUInstallDirs.
Call Stack (most recent call first):
  /llvm/llvm/cmake/modules/AddLLVM.cmake:1 (include)
  CMakeLists.txt:155 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Linker detection: GNU ld
-- Performing Test C_SUPPORTS_FPIC
-- Performing Test C_SUPPORTS_FPIC - Failed
-- Performing Test CXX_SUPPORTS_FPIC
-- Performing Test CXX_SUPPORTS_FPIC - Failed
CMake Warning at /llvm/llvm/cmake/modules/HandleLLVMOptions.cmake:285 (message):
  -fPIC is not supported.
Call Stack (most recent call first):
  /llvm/llvm/cmake/modules/HandleLLVMOptions.cmake:342 (add_flag_or_print_warning)
  CMakeLists.txt:156 (include)

Ultimately it dies because

-- Performing Test CXX_SUPPORTS_FNO_EXCEPTIONS_FLAG - Failed
-- Performing Test CXX_SUPPORTS_FNO_RTTI_FLAG
-- Performing Test CXX_SUPPORTS_FNO_RTTI_FLAG - Failed
CMake Error at /llvm/libunwind/src/CMakeLists.txt:90 (message):
  Compiler doesn't support generation of unwind tables if exception support
  is disabled.  Building libunwind DSO with runtime dependency on C++ ABI
  library is not supported.

but this is misleading: every CXX_SUPPORTS_* compile test has failed due to the unknown architecture.

Exactly as in the original report two-and-a-half years ago,

/llvm/build # bin/clang -dumpmachine
x86_64-unknown-linux-gnu

whereas with explicitly provided -DLLVM_HOST_TRIPLE=x86_64-alpine-linux-musl,

llvm/build # bin/clang -dumpmachine
x86_64-alpine-linux-musl
Endilll commented 11 months ago

Thank you for details!

vindicatorr commented 1 month ago

Yeah, I'm getting this as well when following https://libcxx.llvm.org/BuildingLibcxx.html#bootstrapping-build But setting LLVM_HOST_TRIPLE made no difference in the outcome.

EDIT0: Ah, looking at runtimes/runtimes-<tuple>-bins/CMakeFiles/CMakeConfigureLog.yaml, I saw one of the tests was looking at /usr/lib/Scrt1.o, but for the wrong arch. Currently rebuilding for the build-machine, and I expect it'll be fine. Then I'll do it for the host-target and point to sysroot as well.