llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.09k stars 12k forks source link

s390x build of ClickHouse fails with Clang 18 #95119

Open rschu1ze opened 5 months ago

rschu1ze commented 5 months ago

ClickHouse is a analytical database written in C++. It uses lots of third-party libraries (here), and it is built on multiple platforms including x86, ARM, PowerPC, RISC-V, and s390x.

I bumped the submodule for Google's abseil libraries from a code state of Nov 23 to a code state of Jun 24 here: https://github.com/ClickHouse/ClickHouse/pull/65048

Results of the builds made by ClickHouse's CI are here: https://s3.amazonaws.com/clickhouse-test-reports/65048/1f17ddc6fe35be95736b448ebb3b73123c034196/ci_running.html (click "ClickHouse special build check", then "binary_s390x").

In case the logs are no longer available, I'll paste them here:


2024-06-10T14:24:52.1311439Z Jun 10 16:24:52 FAILED: contrib/abseil-cpp-cmake/CMakeFiles/absl_time.dir/__/abseil-cpp/absl/time/duration.cc.o 
2024-06-10T14:24:52.1321170Z Jun 10 16:24:52 prlimit --as=10000000000 --data=5000000000 --cpu=1000 /usr/bin/sccache /usr/bin/clang++-18 --target=s390x-linux-gnu --sysroot=/build/cmake/linux/../../contrib/sysroot/linux-s390x/s390x-linux-gnu/libc -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -D_LIBUNWIND_IS_NATIVE_ONLY -isystem /build/contrib/abseil-cpp -isystem /build/contrib/llvm-project/libcxx/include -isystem /build/contrib/llvm-project/libcxxabi/include -isystem /build/contrib/libunwind/include --gcc-toolchain=/build/cmake/linux/../../contrib/sysroot/linux-s390x -fdiagnostics-color=always -Xclang -fuse-ctor-homing -Wno-enum-constexpr-conversion -fsized-deallocation  -gdwarf-aranges -pipe -fasynchronous-unwind-tables -ffile-prefix-map=/build=. -ftime-trace -falign-functions=32 -ffp-contract=off  -fdiagnostics-absolute-paths -fstrict-vtable-pointers -w -ffunction-sections -fdata-sections -O2 -g -DNDEBUG -O3 -g  -std=c++23   -D OS_LINUX -Werror -Wno-deprecated-declarations -Wno-poison-system-directories -nostdinc++ -MD -MT contrib/abseil-cpp-cmake/CMakeFiles/absl_time.dir/__/abseil-cpp/absl/time/duration.cc.o -MF contrib/abseil-cpp-cmake/CMakeFiles/absl_time.dir/__/abseil-cpp/absl/time/duration.cc.o.d -o contrib/abseil-cpp-cmake/CMakeFiles/absl_time.dir/__/abseil-cpp/absl/time/duration.cc.o -c /build/contrib/abseil-cpp/absl/time/duration.cc
2024-06-10T14:24:52.1331039Z Jun 10 16:24:52 fatal error: error in backend: Unsupported library call operation!
2024-06-10T14:24:52.1332695Z Jun 10 16:24:52 PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
2024-06-10T14:24:52.1334078Z Jun 10 16:24:52 Stack dump:
2024-06-10T14:24:52.1341580Z Jun 10 16:24:52 0. Program arguments: /usr/bin/clang++-18 -x c++ -c -o contrib/abseil-cpp-cmake/CMakeFiles/absl_time.dir/__/abseil-cpp/absl/time/duration.cc.o -isystem /build/contrib/abseil-cpp -isystem /build/contrib/llvm-project/libcxx/include -isystem /build/contrib/llvm-project/libcxxabi/include -isystem /build/contrib/libunwind/include -nostdinc++ --target=s390x-linux-gnu --sysroot=/build/cmake/linux/../../contrib/sysroot/linux-s390x/s390x-linux-gnu/libc -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -D_LIBUNWIND_IS_NATIVE_ONLY --gcc-toolchain=/build/cmake/linux/../../contrib/sysroot/linux-s390x -fdiagnostics-color=always -Wno-enum-constexpr-conversion -fsized-deallocation -gdwarf-aranges -pipe -fasynchronous-unwind-tables -ffile-prefix-map=/build=. -ftime-trace -falign-functions=32 -ffp-contract=off -fdiagnostics-absolute-paths -fstrict-vtable-pointers -w -ffunction-sections -fdata-sections -O2 -g -DNDEBUG -O3 -g -std=c++23 -DOS_LINUX -Werror -Wno-deprecated-declarations -Wno-poison-system-directories -Xclang -fuse-ctor-homing /build/contrib/abseil-cpp/absl/time/duration.cc
2024-06-10T14:24:52.1348742Z Jun 10 16:24:52 1. <eof> parser at end of file
2024-06-10T14:24:52.1349397Z Jun 10 16:24:52 2. Code generation
2024-06-10T14:24:52.1350342Z Jun 10 16:24:52 3. Running pass 'Function Pass Manager' on module '/build/contrib/abseil-cpp/absl/time/duration.cc'.
2024-06-10T14:24:52.1351716Z Jun 10 16:24:52 4. Running pass 'SystemZ DAG->DAG Pattern Instruction Selection' on function '@_ZN4absl8DurationmLEl'
2024-06-10T14:24:52.1353399Z Jun 10 16:24:52 Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
2024-06-10T14:24:52.1354905Z Jun 10 16:24:52 0  libLLVM.so.18.1      0x00007f70d9453716 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 54
2024-06-10T14:24:52.1355978Z Jun 10 16:24:52 1  libLLVM.so.18.1      0x00007f70d94516d0 llvm::sys::RunSignalHandlers() + 80
2024-06-10T14:24:52.1356890Z Jun 10 16:24:52 2  libLLVM.so.18.1      0x00007f70d93a2fee
2024-06-10T14:24:52.1357496Z Jun 10 16:24:52 3  libLLVM.so.18.1      0x00007f70d93a2fab
2024-06-10T14:24:52.1358312Z Jun 10 16:24:52 4  libLLVM.so.18.1      0x00007f70d944df87 llvm::sys::Process::Exit(int, bool) + 39
2024-06-10T14:24:52.1359173Z Jun 10 16:24:52 5  clang++-18           0x000055c74c829133
2024-06-10T14:24:52.1360063Z Jun 10 16:24:52 6  libLLVM.so.18.1      0x00007f70d93b0832 llvm::report_fatal_error(llvm::Twine const&, bool) + 274
2024-06-10T14:24:52.1360946Z Jun 10 16:24:52 7  libLLVM.so.18.1      0x00007f70d93b0716
2024-06-10T14:24:52.1362630Z Jun 10 16:24:52 8  libLLVM.so.18.1      0x00007f70d9cc12c4 llvm::TargetLowering::makeLibCall(llvm::SelectionDAG&, llvm::RTLIB::Libcall, llvm::EVT, llvm::ArrayRef<llvm::SDValue>, llvm::TargetLowering::MakeLibCallOptions, llvm::SDLoc const&, llvm::SDValue) const + 1908
2024-06-10T14:24:52.1364889Z Jun 10 16:24:52 9  libLLVM.so.18.1      0x00007f70d9d070a7 llvm::TargetLowering::expandMULO(llvm::SDNode*, llvm::SDValue&, llvm::SDValue&, llvm::SelectionDAG&) const + 4039
2024-06-10T14:24:52.1366062Z Jun 10 16:24:52 10 libLLVM.so.18.1      0x00007f70d9b54054
2024-06-10T14:24:52.1366683Z Jun 10 16:24:52 11 libLLVM.so.18.1      0x00007f70d9b4edb9
2024-06-10T14:24:52.1367479Z Jun 10 16:24:52 12 libLLVM.so.18.1      0x00007f70d9b4e54c llvm::SelectionDAG::Legalize() + 364
2024-06-10T14:24:52.1368531Z Jun 10 16:24:52 13 libLLVM.so.18.1      0x00007f70d9ca8d80 llvm::SelectionDAGISel::CodeGenAndEmitDAG() + 1136
2024-06-10T14:24:52.1369823Z Jun 10 16:24:52 14 libLLVM.so.18.1      0x00007f70d9ca7fd2 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) + 4354
2024-06-10T14:24:52.1371553Z Jun 10 16:24:52 15 libLLVM.so.18.1      0x00007f70d9ca618e llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) + 1726
2024-06-10T14:24:52.1372580Z Jun 10 16:24:52 16 libLLVM.so.18.1      0x00007f70dbf52244
2024-06-10T14:24:52.1373635Z Jun 10 16:24:52 17 libLLVM.so.18.1      0x00007f70d9814024 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 852
2024-06-10T14:24:52.1374901Z Jun 10 16:24:52 18 libLLVM.so.18.1      0x00007f70d959a04f llvm::FPPassManager::runOnFunction(llvm::Function&) + 687
2024-06-10T14:24:52.1376141Z Jun 10 16:24:52 19 libLLVM.so.18.1      0x00007f70d959f943 llvm::FPPassManager::runOnModule(llvm::Module&) + 51
2024-06-10T14:24:52.1377302Z Jun 10 16:24:52 20 libLLVM.so.18.1      0x00007f70d959a744 llvm::legacy::PassManagerImpl::run(llvm::Module&) + 1012
2024-06-10T14:24:52.1380836Z Jun 10 16:24:52 21 libclang-cpp.so.18.1 0x00007f70e18ae3ce clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) + 7710
2024-06-10T14:24:52.1384138Z Jun 10 16:24:52 22 libclang-cpp.so.18.1 0x00007f70e1c3d572 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) + 1538
2024-06-10T14:24:52.1385719Z Jun 10 16:24:52 23 libclang-cpp.so.18.1 0x00007f70e086dfc6 clang::ParseAST(clang::Sema&, bool, bool) + 614
2024-06-10T14:24:52.1386964Z Jun 10 16:24:52 24 libclang-cpp.so.18.1 0x00007f70e269ea55 clang::FrontendAction::Execute() + 85
2024-06-10T14:24:52.1388560Z Jun 10 16:24:52 25 libclang-cpp.so.18.1 0x00007f70e2618024 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 708
2024-06-10T14:24:52.1390939Z Jun 10 16:24:52 26 libclang-cpp.so.18.1 0x00007f70e27191fe clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 750
2024-06-10T14:24:52.1393156Z Jun 10 16:24:52 27 clang++-18           0x000055c74c828d55 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) + 4069
2024-06-10T14:24:52.1394560Z Jun 10 16:24:52 28 clang++-18           0x000055c74c826155
2024-06-10T14:24:52.1395226Z Jun 10 16:24:52 29 libclang-cpp.so.18.1 0x00007f70e22d05c9
2024-06-10T14:24:52.1396224Z Jun 10 16:24:52 30 libLLVM.so.18.1      0x00007f70d93a2f8c llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) + 140
2024-06-10T14:24:52.1398581Z Jun 10 16:24:52 31 libclang-cpp.so.18.1 0x00007f70e22cff5e clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const + 366
2024-06-10T14:24:52.1401039Z Jun 10 16:24:52 32 libclang-cpp.so.18.1 0x00007f70e22984d1 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const + 897
2024-06-10T14:24:52.1403395Z Jun 10 16:24:52 33 libclang-cpp.so.18.1 0x00007f70e229871e clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const + 142
2024-06-10T14:24:52.1405871Z Jun 10 16:24:52 34 libclang-cpp.so.18.1 0x00007f70e22b4d2d clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) + 333
2024-06-10T14:24:52.1407713Z Jun 10 16:24:52 35 clang++-18           0x000055c74c825ac4 clang_main(int, char**, llvm::ToolContext const&) + 11172
2024-06-10T14:24:52.1408713Z Jun 10 16:24:52 36 clang++-18           0x000055c74c832bf6 main + 102
2024-06-10T14:24:52.1409351Z Jun 10 16:24:52 37 libc.so.6            0x00007f70d828ed90
2024-06-10T14:24:52.1410449Z Jun 10 16:24:52 38 libc.so.6            0x00007f70d828ee40 __libc_start_main + 128
2024-06-10T14:24:52.1411605Z Jun 10 16:24:52 39 clang++-18           0x000055c74c822bc5 _start + 37
2024-06-10T14:24:52.1413013Z Jun 10 16:24:52 clang++-18: error: clang frontend command failed with exit code 70 (use -v to see invocation)
2024-06-10T14:24:52.1414572Z Jun 10 16:24:52 Ubuntu clang version 18.1.6 (++20240517093811+3d0752b9492e-1~exp1~20240517213934.128)
2024-06-10T14:24:52.1415666Z Jun 10 16:24:52 Target: s390x-unknown-linux-gnu
2024-06-10T14:24:52.1416532Z Jun 10 16:24:52 Thread model: posix
2024-06-10T14:24:52.1417101Z Jun 10 16:24:52 InstalledDir: /usr/bin
2024-06-10T14:24:52.1417919Z Jun 10 16:24:52 clang++-18: note: diagnostic msg: 
2024-06-10T14:24:52.1418765Z Jun 10 16:24:52 ********************
2024-06-10T14:24:52.1419289Z Jun 10 16:24:52 
2024-06-10T14:24:52.1419974Z Jun 10 16:24:52 PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
2024-06-10T14:24:52.1421235Z Jun 10 16:24:52 Preprocessed source(s) and associated run script(s) are located at:
2024-06-10T14:24:52.1422814Z Jun 10 16:24:52 clang++-18: note: diagnostic msg: /tmp/duration-4e964b.cpp
2024-06-10T14:24:52.1424295Z Jun 10 16:24:52 clang++-18: note: diagnostic msg: /tmp/duration-4e964b.sh
2024-06-10T14:24:52.1425407Z Jun 10 16:24:52 clang++-18: note: diagnostic msg: 
2024-06-10T14:24:52.1425889Z Jun 10 16:24:52 
2024-06-10T14:24:52.1426196Z Jun 10 16:24:52 ********************

So this looks like a compiler error ... I was able to bisect the changes in abseil that caused this and it turned out to be this innocent little commit: https://github.com/abseil/abseil-cpp/commit/34604d5b1f6ae14c65b3992478b59f7108051979

The error only happens on s390, builds on all other platforms are okay.

A reproducible example using compiler explorer may be difficult, let me share how I was able to reproduce this locally:

Running CMake, then ninja will fail soon-ish with above error.

Kindly let me know if I can help to analyze this further, thanks.

v01dXYZ commented 5 months ago

Hi, If you have built locally, could you provide the diagnostic files ?

v01dXYZ commented 5 months ago

I was able to get your failure and the diagnostic files. It seems related to Ubuntu which set the default cpu to z13.

95116_diagnostic_files.zip

The diagnostics files are actually pretty useless. I was able to get the same result with the following LLVM IR:

; llc-18 -mcpu=z13 -mtriple=s390x-linux-gnu

define i1 @f(i128 %n.128, i128 %m.128) local_unnamed_addr #0 {
  ; %n.128 = zext i64 %n to i128
  %mul = tail call { i128, i1 } @llvm.umul.with.overflow.i128(i128 %n.128, i128 %m.128)
  %mul.ov = extractvalue { i128, i1 } %mul, 1
  ret i1 %mul.ov
}

; Function Attrs: nocallback nofree nosync nounwind speculatable willreturn memory(none)
declare { i128, i1 } @llvm.umul.with.overflow.i128(i128, i128) #1

attributes #0 = { mustprogress nofree nosync nounwind willreturn memory(none) }
attributes #1 = { nocallback nofree nosync nounwind speculatable willreturn memory(none) }

On both debian and ubuntu I get the failure.

v01dXYZ commented 5 months ago

After bisection, the first bad commit is a65ccc1b9.

v01dXYZ commented 5 months ago

It seems the commit a356e6cc fixes the issue at least on the surface. It changes the implementation of expandMULO. I don t understant what s going on yet. Need some time. Maybe it provides a path to express MUL i128 as i64 ops (which is also done by abseil itself in int128.h).

also the abseil commit causes a regression because it replaces a reference to an external symbol to a constexpr equals to -1 (i128), which allows foldMultiplicationOverflowCheck to match.

v01dXYZ commented 5 months ago

z13 adds new SIMD units with 128-bit vector registers. It seems to make i128 a legal type, but the expansion for MULO didn't support the MULO i128, i128 case until recently.

@rschu1ze The quick and easy fix to your problem is to explicitly set a model anterior to z13 until you can drop it when you switch to LLVM >= 19.

FIY:

rschu1ze commented 4 months ago

Thanks a lot for checking.

The issue is actually not a problem for us, we'll wait till LLVM/Clang >= 19.