llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.07k stars 11.99k forks source link

Crash - LLVM ERROR: Cannot select: intrinsic %llvm.x86.sse42.crc32.32.32 #93426

Open stellarpower opened 5 months ago

stellarpower commented 5 months ago

Seems the same as #15319, although this is occurring "in the wild".

I'm trying to build a large application project (original sources here, but modified since then), so, I'm not manually making use of the CRC functions (it's coming out of ZLib, although previously saw it coming out of Qt somehow), and I expect it would take quite some work to cut it back into an MRE unless someone else has an idea of what the issue might be.

/usr/bin/clang++-18 -target x86_64-linux-gnu -fuse-ld=/usr/bin/ld.lld-18                                 \
    -static-libstdc++ -static-libgcc -flto=full -O0 -g -Wl,--gc-sections -DLLVM_PARALLEL_LINK_JOBS=1     \
    ... libraries here ...                                                                               \
-o bin/linux-x64/Studio

LLVM ERROR: Cannot select: intrinsic %llvm.x86.sse42.crc32.32.32
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Running pass 'Function Pass Manager' on module '../../deps/build/make/lib/linux-x64/zlib.a(deflate.o at 121040)'.
1.      Running pass 'X86 DAG->DAG Instruction Selection' on function '@deflate_fast'
 #0 0x00007f18eb0ecb46 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd93b46)
 #1 0x00007f18eb0eaaf0 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd91af0)
 #2 0x00007f18eb0ed210 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd94210)
 #3 0x00007f18f1a2a420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420)
 #4 0x00007f18e9fc800b raise /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:51:1
 #5 0x00007f18e9fa7859 abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81:7
 #6 0x00007f18eb04a00c llvm::report_fatal_error(llvm::Twine const&, bool) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xcf100c)
 #7 0x00007f18eb93e000 llvm::SelectionDAGISel::isOrEquivalentToAdd(llvm::SDNode const*) const (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15e5000)
 #8 0x00007f18eb93d488 llvm::SelectionDAGISel::SelectCodeCommon(llvm::SDNode*, unsigned char const*, unsigned int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15e4488)
 #9 0x00007f18edda8e63 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3a4fe63)
#10 0x00007f18eb934c3f llvm::SelectionDAGISel::DoInstructionSelection() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15dbc3f)
#11 0x00007f18eb934321 llvm::SelectionDAGISel::CodeGenAndEmitDAG() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15db321)
#12 0x00007f18eb933442 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15da442)
#13 0x00007f18eb9315fe llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15d85fe)
#14 0x00007f18edd9ec6f (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3a45c6f)
#15 0x00007f18eb4a83b4 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x114f3b4)
#16 0x00007f18eb232eff llvm::FPPassManager::runOnFunction(llvm::Function&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xed9eff)
#17 0x00007f18eb2387f3 llvm::FPPassManager::runOnModule(llvm::Module&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xedf7f3)
#18 0x00007f18eb2335f4 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xeda5f4)
#19 0x00007f18ec6c7fe4 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236efe4)
#20 0x00007f18ec6c8f0c (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236ff0c)
#21 0x00007f18ec6c8d8f llvm::lto::thinBackend(llvm::lto::Config const&, unsigned int, std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream, std::default_delete<llvm::CachedFileStream> > > (unsigned int, llvm::Twine const&)>, llvm::Module&, llvm::ModuleSummaryIndex const&, llvm::DenseMap<llvm::StringRef, std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long> >, llvm::DenseMapInfo<llvm::StringRef, void>, llvm::detail::DenseMapPair<llvm::StringRef, std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long> > > > const&, llvm::DenseMap<unsigned long, llvm::GlobalValueSummary*, llvm::DenseMapInfo<unsigned long, void>, llvm::detail::DenseMapPair<unsigned long, llvm::GlobalValueSummary*> > const&, llvm::MapVector<llvm::StringRef, llvm::BitcodeModule, llvm::DenseMap<llvm::StringRef, unsigned int, llvm::DenseMapInfo<llvm::StringRef, void>, llvm::detail::DenseMapPair<llvm::StringRef, unsigned int> >, llvm::SmallVector<std::pair<llvm::StringRef, llvm::BitcodeModule>, 0u> >*, std::vector<unsigned char, std::allocator<unsigned char> > const&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236fd8f)
#22 0x00007f18ec6c2815 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x2369815)
#23 0x00007f18ec6c1fa2 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x2368fa2)
#24 0x00007f18eb017244 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xcbe244)
#25 0x00007f18eb098392 llvm::ThreadPool::processTasks(llvm::ThreadPoolTaskGroup*) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd3f392)
#26 0x00007f18eb098f57 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd3ff57)
#27 0x00007f18f1a1e609 start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478:7
#28 0x00007f18ea0a4133 clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97:0
clang++-18: error: unable to execute command: Aborted (core dumped)
clang++-18: error: linker command failed due to signal (use -v to see invocation)

What's interesting is I have tried cleaning and explicitly turning off using -mno-sse4.2 as wel las explicitly enabling using -msse4.2. The processor I am building on is recent and should support these so I'm not sure why it's unable to output those instructions.

Can try to provide more information, but as mentioned - not sure what would be the most helpful given the size of the project - the build is all spread across tonnes of makefiles with the flags hardcoded, so very hard to follow what options are really being used when.

My application builds using Clang 10 in the Ubuntu repositories; I had upgraded using the PPA to get better C++20 support. I see the same issue with Clang versions 18 and 19; not yet tested any others.

clang-18 --version
Ubuntu clang version 18.1.6 (++20240518023231+1118c2e05e67-1~exp1~20240518143320.131)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
root@localhost /w/F/b/make (master)# dpkg -l | grep -i 18

ii  clang-18                               1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        C, C++ and Objective-C compiler
ii  libclang-common-18-dev:amd64           1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        Clang library - Common development package
ii  libclang-cpp18                         1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        C++ interface to the Clang library
ii  libclang-rt-18-dev:amd64               1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        Compiler-rt - development package
ii  libclang1-18                           1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        C interface to the Clang library
ii  libllvm18:amd64                        1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        Modular compiler and toolchain technologies, runtime library
ii  libmm-glib0:amd64                      1.18.6-1~ubuntu20.04.1                                           amd64        D-Bus service for managing modems - shared libraries
ii  libomp-18-dev                          1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        LLVM OpenMP runtime - dev package
ii  libomp5-18:amd64                       1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64        LLVM OpenMP runtime

-line tool to access X clipboard and selection buffers
root@localhost /w/F/b/make (master)# uname -a
Linux localhost 6.2.0-27-generic #28-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul 12 22:39:51 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
root@localhost /w/F/b/make (master)# apt-cache policy clang-18
clang-18:
  Installed: 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131
  Candidate: 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131
  Version table:
 *** 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 500
        500 http://apt.llvm.org/focal llvm-toolchain-focal-18/main amd64 Packages
        100 /var/lib/dpkg/status
root@localhost /w/F/b/make (master)# 
llvmbot commented 5 months ago

@llvm/issue-subscribers-backend-x86

Author: None (stellarpower)

Seems the same as #15319, although this is occurring "in the wild". I'm trying to build a large application project (original sources [here](https://github.com/neuromore/studio/), but modified since then), so, I'm not manually making use of the CRC functions (it's coming out of ZLib, although previously saw it coming out of Qt somehow), and I expect it would take quite some work to cut it back into an MRE unless someone else has an idea of what the issue might be. ``` /usr/bin/clang++-18 -target x86_64-linux-gnu -fuse-ld=/usr/bin/ld.lld-18 \ -static-libstdc++ -static-libgcc -flto=full -O0 -g -Wl,--gc-sections -DLLVM_PARALLEL_LINK_JOBS=1 \ ... libraries here ... \ -o bin/linux-x64/Studio LLVM ERROR: Cannot select: intrinsic %llvm.x86.sse42.crc32.32.32 PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Running pass 'Function Pass Manager' on module '../../deps/build/make/lib/linux-x64/zlib.a(deflate.o at 121040)'. 1. Running pass 'X86 DAG->DAG Instruction Selection' on function '@deflate_fast' #0 0x00007f18eb0ecb46 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd93b46) #1 0x00007f18eb0eaaf0 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd91af0) #2 0x00007f18eb0ed210 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd94210) #3 0x00007f18f1a2a420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420) #4 0x00007f18e9fc800b raise /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:51:1 #5 0x00007f18e9fa7859 abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81:7 #6 0x00007f18eb04a00c llvm::report_fatal_error(llvm::Twine const&, bool) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xcf100c) #7 0x00007f18eb93e000 llvm::SelectionDAGISel::isOrEquivalentToAdd(llvm::SDNode const*) const (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15e5000) #8 0x00007f18eb93d488 llvm::SelectionDAGISel::SelectCodeCommon(llvm::SDNode*, unsigned char const*, unsigned int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15e4488) #9 0x00007f18edda8e63 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3a4fe63) #10 0x00007f18eb934c3f llvm::SelectionDAGISel::DoInstructionSelection() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15dbc3f) #11 0x00007f18eb934321 llvm::SelectionDAGISel::CodeGenAndEmitDAG() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15db321) #12 0x00007f18eb933442 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15da442) #13 0x00007f18eb9315fe llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x15d85fe) #14 0x00007f18edd9ec6f (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x3a45c6f) #15 0x00007f18eb4a83b4 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x114f3b4) #16 0x00007f18eb232eff llvm::FPPassManager::runOnFunction(llvm::Function&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xed9eff) #17 0x00007f18eb2387f3 llvm::FPPassManager::runOnModule(llvm::Module&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xedf7f3) #18 0x00007f18eb2335f4 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xeda5f4) #19 0x00007f18ec6c7fe4 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236efe4) #20 0x00007f18ec6c8f0c (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236ff0c) #21 0x00007f18ec6c8d8f llvm::lto::thinBackend(llvm::lto::Config const&, unsigned int, std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream, std::default_delete<llvm::CachedFileStream> > > (unsigned int, llvm::Twine const&)>, llvm::Module&, llvm::ModuleSummaryIndex const&, llvm::DenseMap<llvm::StringRef, std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long> >, llvm::DenseMapInfo<llvm::StringRef, void>, llvm::detail::DenseMapPair<llvm::StringRef, std::unordered_set<unsigned long, std::hash<unsigned long>, std::equal_to<unsigned long>, std::allocator<unsigned long> > > > const&, llvm::DenseMap<unsigned long, llvm::GlobalValueSummary*, llvm::DenseMapInfo<unsigned long, void>, llvm::detail::DenseMapPair<unsigned long, llvm::GlobalValueSummary*> > const&, llvm::MapVector<llvm::StringRef, llvm::BitcodeModule, llvm::DenseMap<llvm::StringRef, unsigned int, llvm::DenseMapInfo<llvm::StringRef, void>, llvm::detail::DenseMapPair<llvm::StringRef, unsigned int> >, llvm::SmallVector<std::pair<llvm::StringRef, llvm::BitcodeModule>, 0u> >*, std::vector<unsigned char, std::allocator<unsigned char> > const&) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x236fd8f) #22 0x00007f18ec6c2815 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x2369815) #23 0x00007f18ec6c1fa2 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0x2368fa2) #24 0x00007f18eb017244 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xcbe244) #25 0x00007f18eb098392 llvm::ThreadPool::processTasks(llvm::ThreadPoolTaskGroup*) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd3f392) #26 0x00007f18eb098f57 (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd3ff57) #27 0x00007f18f1a1e609 start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478:7 #28 0x00007f18ea0a4133 clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97:0 clang++-18: error: unable to execute command: Aborted (core dumped) clang++-18: error: linker command failed due to signal (use -v to see invocation) ``` What's interesting is I have tried cleaning and explicitly turning off using `-mno-sse4.2` as wel las explicitly enabling using `-msse4.2`. The processor I am building on is recent and should support these so I'm not sure why it's unable to output those instructions. Can try to provide more information, but as mentioned - not sure what would be the most helpful given the size of the project - the build is all spread across tonnes of makefiles with the flags hardcoded, so very hard to follow what options are really being used when. My application builds using Clang 10 in the Ubuntu repositories; I had upgraded using the PPA to get better C++20 support. I see the same issue with Clang versions 18 and 19; not yet tested any others. ``` clang-18 --version Ubuntu clang version 18.1.6 (++20240518023231+1118c2e05e67-1~exp1~20240518143320.131) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin root@localhost /w/F/b/make (master)# dpkg -l | grep -i 18 ii clang-18 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 C, C++ and Objective-C compiler ii libclang-common-18-dev:amd64 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 Clang library - Common development package ii libclang-cpp18 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 C++ interface to the Clang library ii libclang-rt-18-dev:amd64 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 Compiler-rt - development package ii libclang1-18 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 C interface to the Clang library ii libllvm18:amd64 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 Modular compiler and toolchain technologies, runtime library ii libmm-glib0:amd64 1.18.6-1~ubuntu20.04.1 amd64 D-Bus service for managing modems - shared libraries ii libomp-18-dev 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 LLVM OpenMP runtime - dev package ii libomp5-18:amd64 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 amd64 LLVM OpenMP runtime -line tool to access X clipboard and selection buffers root@localhost /w/F/b/make (master)# uname -a Linux localhost 6.2.0-27-generic #28-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul 12 22:39:51 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux root@localhost /w/F/b/make (master)# apt-cache policy clang-18 clang-18: Installed: 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 Candidate: 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 Version table: *** 1:18.1.6~++20240518023231+1118c2e05e67-1~exp1~20240518143320.131 500 500 http://apt.llvm.org/focal llvm-toolchain-focal-18/main amd64 Packages 100 /var/lib/dpkg/status root@localhost /w/F/b/make (master)# ```
phoebewang commented 5 months ago

What's interesting is I have tried cleaning and explicitly turning off using -mno-sse4.2 as wel las explicitly enabling using -msse4.2. The processor I am building on is recent and should support these so I'm not sure why it's unable to output those instructions.

Why do you want to turn off SSE4.2?

topperc commented 5 months ago

CRC32 has a separate enable control from -msse4.2 too. -mcrc32 and -mno-crc32. Clang is supposed to enable it with -msse4.2 (or a SSE4.2 cpu) as long as -mno-crc32 is not specified.

I wonder if somehow the "crc32" feature is not set for that function when the IR reaches the backend, but I'm not sure how that would happen.

stellarpower commented 5 months ago

What's interesting is I have tried cleaning and explicitly turning off using -mno-sse4.2 as wel las explicitly enabling using -msse4.2. The processor I am building on is recent and should support these so I'm not sure why it's unable to output those instructions.

Why do you want to turn off SSE4.2?

I don't in general, it was just in an effort to try and avoid this error. I'll try the crc32 flags as topper mentions and see what happens. The original build seems to be using link-time optimisation, so I don't know if that has anything to do with the issue. The dependencies in general will have been built using different flags to the main project, although obviously this probably shouldn't cause an internal crash.

phoebewang commented 5 months ago

It's normal to see internal crash when using target specific intrinsics but lacking related features in LLVM IR. But the FE should have checked for all the cases, and error out it early. I don't see any problem with these crc32 intrinsic definations in the FE, so it's still a mystery how the IR being generated.

stellarpower commented 5 months ago

Agreed, why it can't select them is possibly due to the specifics of this big heap of Unix makefiles used to generate the project, but, it's probably best it doesn't crash internally, and also, if it is able to produce an error message, including diagnostic information about why the instruction couldn't be selected would probably also be useful and then if there is a problem in my setup, I can go and fix that. I also haven't pinned down the exact major version where this changed, but clang 10 builds it without a problem. Whilst I assume there will have obviously been pretty major backend changes since then, the crash is new in a sense at least.

phoebewang commented 5 months ago

It may be affected by https://github.com/llvm/llvm-project/commit/12fa608af44a80de8b655a8a984cd095908e7e80, which included since LLVM14, if it was good in an earlier version. But I cannot tell what's wrong without a reasonable reproducer. It would be helpful if you can provide a small reproducer.