Backend code generator crash while running pass "X86 DAG->DAG Instruction Selection". Maybe related to coroutines.

[atulkatti]

We are seeing this crash in official builds of Edge. This repro has been provided using internal LLVM build that should be compatible with the following Chromium upstream LLVM build: CLANG_REVISION = 'llvmorg-20-init-1009-g7088a5ed' CLANG_SUB_REVISION = 9

This actively blocks our consumption of the latest LLVM bits, so it will help if this gets looked at soon. We had the exact same break during Chromium clang roll in May-2024 and it got fixed with the next Clang roll in June-2024. It is broken again with the latest August-2024 Clang roll referenced above. We are wondering if this is related to churn in the coroutines codebase. Attaching the reduced repro source file if this can be included in your test suite so it can prevent future breaks in this area.

The crash reproduces with the LLC command line. Bugpoint crashed during reduction. llc.exe coroutines_main.bc

Please find both the reduced repro source file and the .bc file (attached as .bc.txt). coroutines_main.bc.txt Reduce repro source file main.cc.txt

$src\out\official_x64>$src\third_party\llvm-build\Release+Asserts\bin\llc.exe coroutines_main.bc
LLVM ERROR: Do not know how to promote this operator!
PLEASE submit a bug report to https://crbug.com in the Tools>LLVM component, run tools/clang/scripts/process_crashreports.py (only if inside Google) to upload crash related files, and include the crash backtrace.
Stack dump:
0.      Program arguments: third_party\\llvm-build\\Release+Asserts\\bin\\llc.exe coroutines_main.bc
1.      Running pass 'Function Pass Manager' on module 'coroutines_main.bc'.
2.      Running pass 'X86 DAG->DAG Instruction Selection' on function '@"?Deploy@@YA?AUTacticalNuke@@XZ"'
Exception Code: 0xC000001D
 #0 0x00007ff77d2e95e6 HandleAbort $src\third_party\llvm\llvm\lib\Support\Windows\Signals.inc:429:0
 #1 0x00007ff77e082ad2 raise $src\third_party\llvm-build\Release+Asserts\minkernel\crts\ucrt\src\appcrt\misc\signal.cpp:547:0
 #2 0x00007ff77e07a8e8 abort $src\third_party\llvm-build\Release+Asserts\minkernel\crts\ucrt\src\appcrt\startup\abort.cpp:71:0
 #3 0x00007ff77c17537d llvm::report_fatal_error(class llvm::Twine const &, bool) $src\third_party\llvm\llvm\lib\Support\ErrorHandling.cpp:125:0
 #4 0x00007ff77c175165 llvm::report_fatal_error(char const *, bool) $src\third_party\llvm\llvm\lib\Support\ErrorHandling.cpp:83:0
 #5 0x00007ff77de1d173 llvm::DAGTypeLegalizer::PromoteIntegerResult(class llvm::SDNode *, unsigned int) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeIntegerTypes.cpp:57:0
 #6 0x00007ff77d408e5e llvm::DAGTypeLegalizer::run(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeTypes.cpp:261:0
 #7 0x00007ff77d40c54e llvm::SelectionDAG::LegalizeTypes(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeTypes.cpp:1057:0
 #8 0x00007ff77c0f7cc7 llvm::SelectionDAGISel::CodeGenAndEmitDAG(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:933:0
 #9 0x00007ff77c0f7b22 llvm::SelectionDAGISel::SelectBasicBlock(class llvm::ilist_iterator_w_bits<struct llvm::ilist_detail::node_options<class llvm::Instruction, 0, 0, void, 1, class llvm::BasicBlock>, 0, 1>, class llvm::ilist_iterator_w_bits<struct llvm::ilist_detail::node_options<class llvm::Instruction, 0, 0, void, 1, class llvm::BasicBlock>, 0, 1>, bool &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:831:0
#10 0x00007ff77c0f748d llvm::SelectionDAGISel::SelectAllBasicBlocks(class llvm::Function const &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:1599:0
#11 0x00007ff77c0f54da llvm::SelectionDAGISel::runOnMachineFunction(class llvm::MachineFunction &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:615:0
#12 0x00007ff77c96ed6d `anonymous namespace'::X86DAGToDAGISel::runOnMachineFunction $src\third_party\llvm\llvm\lib\Target\X86\X86ISelDAGToDAG.cpp:190:0
#13 0x00007ff77c0f434b llvm::SelectionDAGISelLegacy::runOnMachineFunction(class llvm::MachineFunction &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:374:0
#14 0x00007ff77c280d84 llvm::MachineFunctionPass::runOnFunction(class llvm::Function &) $src\third_party\llvm\llvm\lib\CodeGen\MachineFunctionPass.cpp:94:0
#15 0x00007ff77bf61ca9 llvm::FPPassManager::runOnFunction(class llvm::Function &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1442:0
#16 0x00007ff77bf67df3 llvm::FPPassManager::runOnModule(class llvm::Module &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1488:0
#17 0x00007ff77bf62450 `anonymous namespace'::MPPassManager::runOnModule $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1557:0
#18 0x00007ff77bf620ae llvm::legacy::PassManagerImpl::run(class llvm::Module &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:542:0
#19 0x00007ff77bd659a6 compileModule $src\third_party\llvm\llvm\tools\llc\llc.cpp:742:0
#20 0x00007ff77bd63c1d main $src\third_party\llvm\llvm\tools\llc\llc.cpp:409:0
#21 0x00007ff77e06b910 invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
#22 0x00007ff77e06b910 __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#23 0x00007fff69c17374 (C:\WINDOWS\System32\KERNEL32.DLL+0x17374)
#24 0x00007fff69d5cc91 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x4cc91)

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: atul (atulkatti)

We are seeing this crash in official builds of Edge. This repro has been provided using internal LLVM build that should be compatible with the following Chromium upstream LLVM build: CLANG_REVISION = 'llvmorg-20-init-1009-g7088a5ed' CLANG_SUB_REVISION = 9 This actively blocks our consumption of the latest LLVM bits, so it will help if this gets looked at soon. We had the exact same break during Chromium clang roll in May-2024 and it got fixed with the next Clang roll in June-2024. It is broken again with the latest August-2024 Clang roll referenced above. We are wondering if this is related to churn in the coroutines codebase. Attaching the reduced repro source file if this can be included in your test suite so it can prevent future breaks in this area. The crash reproduces with the LLC command line. Bugpoint crashed during reduction. **llc.exe coroutines_main.bc** Please find both the reduced repro source file and the .bc file (attached as .bc.txt). [coroutines_main.bc.txt](https://github.com/user-attachments/files/16631162/coroutines_main.bc.txt) [Reduce repro source file main.cc.txt](https://github.com/user-attachments/files/16631233/main.cc.txt) $src\out\official_x64>$src\third_party\llvm-build\Release+Asserts\bin\llc.exe coroutines_main.bc LLVM ERROR: Do not know how to promote this operator! PLEASE submit a bug report to https://crbug.com in the Tools>LLVM component, run tools/clang/scripts/process_crashreports.py (only if inside Google) to upload crash related files, and include the crash backtrace. Stack dump: 0. Program arguments: third_party\\llvm-build\\Release+Asserts\\bin\\llc.exe coroutines_main.bc 1. Running pass 'Function Pass Manager' on module 'coroutines_main.bc'. 2. Running pass 'X86 DAG->DAG Instruction Selection' on function '@"?Deploy@@YA?AUTacticalNuke@@XZ"' Exception Code: 0xC000001D #0 0x00007ff77d2e95e6 HandleAbort $src\third_party\llvm\llvm\lib\Support\Windows\Signals.inc:429:0 #1 0x00007ff77e082ad2 raise $src\third_party\llvm-build\Release+Asserts\minkernel\crts\ucrt\src\appcrt\misc\signal.cpp:547:0 #2 0x00007ff77e07a8e8 abort $src\third_party\llvm-build\Release+Asserts\minkernel\crts\ucrt\src\appcrt\startup\abort.cpp:71:0 #3 0x00007ff77c17537d llvm::report_fatal_error(class llvm::Twine const &, bool) $src\third_party\llvm\llvm\lib\Support\ErrorHandling.cpp:125:0 #4 0x00007ff77c175165 llvm::report_fatal_error(char const *, bool) $src\third_party\llvm\llvm\lib\Support\ErrorHandling.cpp:83:0 #5 0x00007ff77de1d173 llvm::DAGTypeLegalizer::PromoteIntegerResult(class llvm::SDNode *, unsigned int) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeIntegerTypes.cpp:57:0 #6 0x00007ff77d408e5e llvm::DAGTypeLegalizer::run(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeTypes.cpp:261:0 #7 0x00007ff77d40c54e llvm::SelectionDAG::LegalizeTypes(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\LegalizeTypes.cpp:1057:0 #8 0x00007ff77c0f7cc7 llvm::SelectionDAGISel::CodeGenAndEmitDAG(void) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:933:0 #9 0x00007ff77c0f7b22 llvm::SelectionDAGISel::SelectBasicBlock(class llvm::ilist_iterator_w_bits<struct llvm::ilist_detail::node_options<class llvm::Instruction, 0, 0, void, 1, class llvm::BasicBlock>, 0, 1>, class llvm::ilist_iterator_w_bits<struct llvm::ilist_detail::node_options<class llvm::Instruction, 0, 0, void, 1, class llvm::BasicBlock>, 0, 1>, bool &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:831:0 #10 0x00007ff77c0f748d llvm::SelectionDAGISel::SelectAllBasicBlocks(class llvm::Function const &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:1599:0 #11 0x00007ff77c0f54da llvm::SelectionDAGISel::runOnMachineFunction(class llvm::MachineFunction &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:615:0 #12 0x00007ff77c96ed6d `anonymous namespace'::X86DAGToDAGISel::runOnMachineFunction $src\third_party\llvm\llvm\lib\Target\X86\X86ISelDAGToDAG.cpp:190:0 #13 0x00007ff77c0f434b llvm::SelectionDAGISelLegacy::runOnMachineFunction(class llvm::MachineFunction &) $src\third_party\llvm\llvm\lib\CodeGen\SelectionDAG\SelectionDAGISel.cpp:374:0 #14 0x00007ff77c280d84 llvm::MachineFunctionPass::runOnFunction(class llvm::Function &) $src\third_party\llvm\llvm\lib\CodeGen\MachineFunctionPass.cpp:94:0 #15 0x00007ff77bf61ca9 llvm::FPPassManager::runOnFunction(class llvm::Function &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1442:0 #16 0x00007ff77bf67df3 llvm::FPPassManager::runOnModule(class llvm::Module &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1488:0 #17 0x00007ff77bf62450 `anonymous namespace'::MPPassManager::runOnModule $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:1557:0 #18 0x00007ff77bf620ae llvm::legacy::PassManagerImpl::run(class llvm::Module &) $src\third_party\llvm\llvm\lib\IR\LegacyPassManager.cpp:542:0 #19 0x00007ff77bd659a6 compileModule $src\third_party\llvm\llvm\tools\llc\llc.cpp:742:0 #20 0x00007ff77bd63c1d main $src\third_party\llvm\llvm\tools\llc\llc.cpp:409:0 #21 0x00007ff77e06b910 invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0 #22 0x00007ff77e06b910 __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0 #23 0x00007fff69c17374 (C:\WINDOWS\System32\KERNEL32.DLL+0x17374) #24 0x00007fff69d5cc91 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x4cc91)

[lukel97]

Can you check if this was fixed in e027e04f0139126b0a4af3aba0dbb31fdf22ea62?

[RKSimon]

Legalizing node: t14: i1,ch = llvm.coro.alloc t12:1, TargetConstant:i64<27>, t12, ../../nwc/main.cc:19
Analyzing result type: i1
Promote integer result: t14: i1,ch = llvm.coro.alloc t12:1, TargetConstant:i64<27>, t12, ../../nwc/main.cc:19
PromoteIntegerResult #0: t14: i1,ch = llvm.coro.alloc t12:1, TargetConstant:i64<27>, t12, ../../nwc/main.cc:19

LLVM ERROR: Do not know how to promote this operator!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.  Program arguments: ninja\\bin\\llc coroutines_main.bc -o bar.s -debug
1.  Running pass 'Function Pass Manager' on module 'coroutines_main.bc'.
2.  Running pass 'X86 DAG->DAG Instruction Selection' on function '@"?Deploy@@YA?AUTacticalNuke@@XZ"'

[lukel97]

I presume g7088a5ed is from commit 7088a5ed, which is before 234cb4c6e3a382a5c0b3396647a1839699944ec0. So it's probably not related

[topperc]

I don't think llvm.coro.alloc is supposed to reach the backend. The CoroCleanup pass should remove it in the middle end.

[topperc]

There's a commit that mention CoroCleanup from the end of July

commit 3a9ef4e69a3fec3203bd3e1caa53edf4b76843cf
Author: Wei Wang <apollo.mobility@gmail.com>
Date:   Mon Jul 29 17:42:01 2024

    [Pipelines] Do not run CoroSplit and CoroCleanup in LTO pre-link pipeline (#100205)

    This is re-land of #90310 after making asan skip pre-split coroutines in
    #99415.

    Skip CoroSplit and CoroCleanup in LTO pre-link pipeline so that
    CoroElide can happen after callee coroutine is imported into caller's
    module in ThinLTO.

@atulkatti are you using ThinLTO?

[atulkatti]

@topperc Yes, we are using ThinLTO. But on the reduced repro I was able to reproduce the issue even when LTO was disabled.

[atulkatti]

@topperc Yes, we are using ThinLTO. But on the reduced repro I was able to reproduce the issue even when LTO was disabled.

@topperc Correction, my earlier attempt to disable ThinLTO didn't work as expected due to flags being added elsewhere in the build system. Once I disabled ThinLTO correctly the crash went away. However, we can't disable ThinLTO completely in our product build. Would it be possible to make a fix for this issue when ThinLTO is being used?

[topperc]

@apolloww Can you take a look at this?

[apolloww]

The change moves both CoroSplit and CoroCleanup into ThinLTO post link optimization pipeline, but CoroCleanup should still run before going into codegen passes. If the pre-link compilation has -flto=thin flag, the output bitcode retains all coroutine intrinsics, including llvm.coro.alloc. The bitcode file is meant to be fed into ThinLTO compilation. Otherwise, all coroutine intrinsics should be lowered.

Could you share how coroutines_main.bc is generated? Just checked the bitcode file, it has module summary, so likely compiled with -flto=thin. Then llc invokes codegen passes without calling opt passes. If ThinLTO is not going to be applied to the bitcode module, you can just remove the -flto=thin flag.

[atulkatti]

@apolloww The repro file I shared is from a reduced repro of the real issue we are facing in Edge official build where ThinLTO is enabled. The crashing code exists in a first party library that we consume and we can't disable ThinLTO due to performance implications in important scenarios. Can a fix be made so that the intrinsics get lowered before codegen when ThinLTO is enabled?

[apolloww]

@apolloww The repro file I shared is from a reduced repro of the real issue we are facing in Edge official build where ThinLTO is enabled. The crashing code exists in a first party library that we consume and we can't disable ThinLTO due to performance implications in important scenarios. Can a fix be made so that the intrinsics get lowered before codegen when ThinLTO is enabled?

Could you share the flags that are passed into pre-link compilation and linking? If every step is invoked via clang driver, the intrinsics should be lowered before entering the CodeGen pipeline. So if you have something like

# compile
clang -flto=thin -O3 -c foo.cpp -o foo.o
clang -flto=thin -O3 -c bar.cpp -o bar.o

#link
clang -flto=thin -fuse-ld=lld foo.o. bar.o -o bin

It should work.

[atulkatti]

@apolloww Thanks for looking into this and providing guidance. After looking into our build logs I was able to confirm that we are indeed disabling ThinLTO during linking. So there is a mismatch in the ThinLTO enablement between the compilation and linking steps. We will try to workaround this issue by enabling ThinLTO for the failing component.

[vitalybuka]

I can reproduce this crash as well, and revert https://github.com/llvm/llvm-project/pull/100205 helps.

[apolloww]

I can reproduce this crash as well, and revert #100205 helps.

Is this still causing asan pipeline failure?

[vitalybuka]

No asan failures, it's not on bot.

This is a different issue, internal build with CFI and ThinLTO.

LLVM ERROR: Cannot select: intrinsic %llvm.coro.subfn.addr
Stack dump:
....
  llvm::report_fatal_error(llvm::Twine const&, bool) llvm-project/llvm/lib/Support/ErrorHandling.cpp:0:5
  llvm::SelectionDAGISel::CannotYetSelect(llvm::SDNode*) llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:4360:47
  (third_party/crosstool/v18/llvm_unstable/toolchain/bin/ld+0x3e1d76a)
  (anonymous namespace)::X86DAGToDAGISel::Select(llvm::SDNode*) llvm-project/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp:0:0
  llvm::operator!=(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::SDNode, false, false, void, false, void>, false, false> const&, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::SDNode, false, false, void, false, void>, false, false> const&) llvm-project/llvm/include/llvm/ADT/ilist_iterator.h:178:16
 llvm::SelectionDAGISel::DoInstructionSelection() llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1237:25
 llvm::SelectionDAGISel::CodeGenAndEmitDAG() llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1076:3
 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1599:8
 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:615:22
 llvm::SelectionDAGISelLegacy::runOnMachineFunction(llvm::MachineFunction&) llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:374:20
 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:94:13
 llvm::FPPassManager::runOnFunction(llvm::Function&) llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1408:27
 llvm::FPPassManager::runOnModule(llvm::Module&) llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1454:13
 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1523:27
 llvm::legacy::PassManagerImpl::run(llvm::Module&) llvm-project/llvm/lib/IR/LegacyPassManager.cpp:541:44
 std::__u::unique_ptr<llvm::ToolOutputFile, std::__u::default_delete<llvm::ToolOutputFile>>::operator bool() const third_party/crosstool/v18/stable/src/libcxx/include/__memory/unique_ptr.h:297:19
 codegen(llvm::lto::Config const&, llvm::TargetMachine*, std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>, unsigned int, llvm::Module&, llvm::ModuleSummaryIndex const&) llvm-project/llvm/lib/LTO/LTOBackend.cpp:432:7
 std::__u::__function::__policy_func<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~__policy_func() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:692:9
 std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~function() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:980:43
 llvm::lto::backend(llvm::lto::Config const&, std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>, unsigned int, llvm::Module&, llvm::ModuleSummaryIndex&) llvm-project/llvm/lib/LTO/LTOBackend.cpp:534:5
 std::__u::__function::__policy_func<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~__policy_func() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:692:9
 std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~function() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:980:43
 llvm::lto::LTO::runRegularLTO(std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>) llvm-project/llvm/lib/LTO/LTO.cpp:1353:13
 std::__u::__function::__policy_func<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~__policy_func() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:692:9
 std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>::~function() third_party/crosstool/v18/stable/src/libcxx/include/__functional/function.h:980:43
 llvm::lto::LTO::run(std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>, std::__u::function<llvm::Expected<std::__u::function<llvm::Expected<std::__u::unique_ptr<llvm::CachedFileStream, std::__u::default_delete<llvm::CachedFileStream>>> (unsigned int, llvm::Twine const&)>> (unsigned int, llvm::StringRef, llvm::Twine const&)>) llvm-project/llvm/lib/LTO/LTO.cpp:1186:18
 lld::elf::BitcodeCompiler::compile() llvm-project/lld/ELF/LTO.cpp:327:5
 std::__u::vector<lld::elf::InputFile*, std::__u::allocator<lld::elf::InputFile*>>::begin() third_party/crosstool/v18/stable/src/libcxx/include/vector:1411:28
 void lld::elf::LinkerDriver::compileBitcodeFiles<llvm::object::ELFType<(llvm::endianness)1, true>>(bool) llvm-project/lld/ELF/Driver.cpp:2521:24
 void lld::elf::LinkerDriver::link<llvm::object::ELFType<(llvm::endianness)1, true>>(llvm::opt::InputArgList&) llvm-project/lld/ELF/Driver.cpp:2994:3
 lld::elf::LinkerDriver::linkerMain(llvm::ArrayRef<char const*>) llvm-project/lld/ELF/Driver.cpp:0:5
 lld::elf::link(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, bool, bool) llvm-project/lld/ELF/Driver.cpp:172:10
 lld::unsafeLldMain(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, llvm::ArrayRef<lld::DriverDef>, bool) llvm-project/lld/Common/DriverDispatcher.cpp:164:11
 lld_main(int, char**, llvm::ToolContext const&) llvm-project/lld/tools/lld/lld.cpp:115:1
 main llvm-project/lld/lld-driver.cpp:17:10

[apolloww]

No asan failures, it's not on bot.

This is a different issue, internal build with CFI and ThinLTO.

OK. I'll put the change behind a flag.

[vitalybuka]

No asan failures, it's not on bot. This is a different issue, internal build with CFI and ThinLTO.

OK. I'll put the change behind a flag.

Flags is LGTM.

However, I didn't look into details, but are there difficulties to support invocation like in the stack trace?

[apolloww]

No asan failures, it's not on bot. This is a different issue, internal build with CFI and ThinLTO.

OK. I'll put the change behind a flag.

Flags is LGTM.

However, I didn't look into details, but are there difficulties to support invocation like in the stack trace?

From the stack trace, it seems the invocation was through regular LTO compilation. I think Coro passes are not enabled in regular LTO post-link pipeline.

Even if we fix this case, there are still cases that user can take the pre-link compilation output and feed directly into codegen pipeline.

107153 puts the change behind a flag.