search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
27.92k stars 11.52k forks source link

Clang 18 crashes if built with PGO #91041

Open vient opened 4 months ago

vient commented 4 months ago

I use tag llvmorg-18.1.5. Build without PGO works fine, then I gathered some IR profiles and tried to rebuild LLVM. It crashed while trying to build runtimes with what looked like the same error. I've taken one example source and ran it through cvise.

The crash does not reproduce with clang built without PGO

Crash:

$ /root/llvm-build/stage2-prof-use/bin/clang++ -cc1 -emit-obj -O1 -vectorize-loops locale-8f8074.cpp
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /root/llvm-build/stage2-prof-use/bin/clang++ -cc1 -emit-obj -O1 -vectorize-loops locale-8f8074.cpp
1.      <eof> parser at end of file
2.      Optimizer
 #0 0x00000000036c5797 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /root/llvm-build/llvm-project/llvm/lib/Support/Unix/Signals.inc:727:8
 #1 0x00000000036c5bde llvm::sys::RunSignalHandlers() /root/llvm-build/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x00000000036c5bde SignalHandler(int) /root/llvm-build/llvm-project/llvm/lib/Support/Unix/Signals.inc:403:3
 #3 0x00007f4716770420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420)
 #4 0x0000000002ad2d07 std::function<llvm::VPValue* (llvm::Value*)>::function(std::function<llvm::VPValue* (llvm::Value*)>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:409:17
 #5 0x0000000002ad2d07 void std::_Construct<std::function<llvm::VPValue* (llvm::Value*)>, std::function<llvm::VPValue* (llvm::Value*)>>(std::function<llvm::VPValue* (llvm::Value*)>*, std::function<llvm::VPValue* (llvm::Value*)>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/stl_construct.h:119:25
 #6 0x0000000002ad2d07 void std::_Optional_payload_base<std::function<llvm::VPValue* (llvm::Value*)>>::_M_construct<std::function<llvm::VPValue* (llvm::Value*)>>(std::function<llvm::VPValue* (llvm::Value*)>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:274:4
 #7 0x0000000002ad2d07 std::_Optional_payload_base<std::function<llvm::VPValue* (llvm::Value*)>>::_Optional_payload_base(bool, std::_Optional_payload_base<std::function<llvm::VPValue* (llvm::Value*)>>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:151:10
 #8 0x0000000002ad2d07 std::_Optional_payload<std::function<llvm::VPValue* (llvm::Value*)>, true, false, false>::_Optional_payload(bool, std::_Optional_payload_base<std::function<llvm::VPValue* (llvm::Value*)>>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:395:42
 #9 0x0000000002ad2d07 std::_Optional_payload<std::function<llvm::VPValue* (llvm::Value*)>, false, false, false>::_Optional_payload(bool, std::_Optional_payload_base<std::function<llvm::VPValue* (llvm::Value*)>>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:429:57
#10 0x0000000002ad2d07 std::_Optional_base<std::function<llvm::VPValue* (llvm::Value*)>, false, false>::_Optional_base(std::_Optional_base<std::function<llvm::VPValue* (llvm::Value*)>, false, false>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:542:9
#11 0x0000000002ad2d07 std::optional<std::function<llvm::VPValue* (llvm::Value*)>>::optional(std::optional<std::function<llvm::VPValue* (llvm::Value*)>>&&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/optional:703:11
#12 0x0000000002ad2d07 llvm::callable_detail::Callable<std::function<llvm::VPValue* (llvm::Value*)>, false>::Callable(llvm::callable_detail::Callable<std::function<llvm::VPValue* (llvm::Value*)>, false>&&) /root/llvm-build/llvm-project/llvm/include/llvm/ADT/STLExtras.h:225:3
#13 0x0000000002ad2d07 llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>::mapped_iterator(llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>&&) /root/llvm-build/llvm-project/llvm/include/llvm/ADT/STLExtras.h:347:7
#14 0x0000000002ad2d07 llvm::iterator_range<llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>> llvm::make_range<llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>>(llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>, llvm::mapped_iterator<llvm::Use*, std::function<llvm::VPValue* (llvm::Value*)>, llvm::VPValue*>) /root/llvm-build/llvm-project/llvm/include/llvm/ADT/iterator_range.h:75:42
#15 0x0000000002ad2d07 auto llvm::map_range<llvm::iterator_range<llvm::Use*>&, std::function<llvm::VPValue* (llvm::Value*)>>(llvm::iterator_range<llvm::Use*>&, std::function<llvm::VPValue* (llvm::Value*)>) /root/llvm-build/llvm-project/llvm/include/llvm/ADT/STLExtras.h:378:10
#16 0x0000000001ed0de8 std::_Function_base::~_Function_base() /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_function.h:243:11
#17 0x0000000001ed0de8 llvm::VPlan::mapToVPValues(llvm::iterator_range<llvm::Use*>) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/VPlan.h:2841:5
#18 0x0000000001ed0de8 llvm::LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(llvm::VFRange&) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8764:30
#19 0x0000000002b906f3 std::__uniq_ptr_impl<llvm::VPlan, std::default_delete<llvm::VPlan>>::_M_ptr() const /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:193:51
#20 0x0000000002b906f3 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::get() const /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:464:21
#21 0x0000000002b906f3 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::operator bool() const /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:481:16
#22 0x0000000002b906f3 llvm::LoopVectorizationPlanner::buildVPlansWithVPRecipes(llvm::ElementCount, llvm::ElementCount) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8600:14
#23 0x0000000002ac306e llvm::LoopVectorizationPlanner::plan(llvm::ElementCount, unsigned int) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:7419:3
#24 0x0000000002aeac7a llvm::LoopVectorizePass::processLoop(llvm::Loop*) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10004:12
#25 0x000000000260c742 llvm::LoopVectorizePass::runImpl(llvm::Function&, llvm::ScalarEvolution&, llvm::LoopInfo&, llvm::TargetTransformInfo&, llvm::DominatorTree&, llvm::BlockFrequencyInfo*, llvm::TargetLibraryInfo*, llvm::DemandedBits&, llvm::AssumptionCache&, llvm::LoopAccessInfoManager&, llvm::OptimizationRemarkEmitter&, llvm::ProfileSummaryInfo*) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10330:30
#26 0x0000000002609391 llvm::LoopVectorizePass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10369:9
#27 0x00000000026090cd llvm::detail::PassModel<llvm::Function, llvm::LoopVectorizePass, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:89:5
#28 0x0000000001cf5bee llvm::SmallPtrSetImplBase::size() const /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:93:35
#29 0x0000000001cf5bee llvm::SmallPtrSetImplBase::empty() const /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:92:45
#30 0x0000000001cf5bee llvm::PreservedAnalyses::allAnalysesInSetPreserved(llvm::AnalysisSetKey*) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManager.h:363:36
#31 0x0000000001cf5bee bool llvm::PreservedAnalyses::allAnalysesInSetPreserved<llvm::AllAnalysesOn<llvm::Function>>() const /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManager.h:356:12
#32 0x0000000001cf5bee llvm::AnalysisManager<llvm::Function>::invalidate(llvm::Function&, llvm::PreservedAnalyses const&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManagerImpl.h:92:10
#33 0x0000000001cf5bee llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManager.h:547:10
#34 0x0000000001cf59dd llvm::detail::PassModel<llvm::Function, llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:89:5
#35 0x0000000001cf5522 llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /root/llvm-build/llvm-project/llvm/lib/IR/PassManager.cpp:128:23
#36 0x0000000001cf531d llvm::detail::PassModel<llvm::Module, llvm::ModuleToFunctionPassAdaptor, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManagerInternal.h:89:5
#37 0x0000000001f1815e llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManager.h:547:10
#38 0x0000000002c670a7 llvm::SmallPtrSetImplBase::isSmall() const /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:195:33
#39 0x0000000002c670a7 llvm::SmallPtrSetImplBase::~SmallPtrSetImplBase() /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallPtrSet.h:83:10
#40 0x0000000002c670a7 llvm::PreservedAnalyses::~PreservedAnalyses() /root/llvm-build/llvm-project/llvm/include/llvm/IR/PassManager.h:172:7
#41 0x0000000002c670a7 (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>&, std::unique_ptr<llvm::ToolOutputFile, std::default_delete<llvm::ToolOutputFile>>&, clang::BackendConsumer*) /root/llvm-build/llvm-project/clang/lib/CodeGen/BackendUtil.cpp:1101:5
#42 0x0000000002a75124 (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) /root/llvm-build/llvm-project/clang/lib/CodeGen/BackendUtil.cpp:0:3
#43 0x0000000002a75124 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) /root/llvm-build/llvm-project/clang/lib/CodeGen/BackendUtil.cpp:1328:13
#44 0x000000000274cc42 std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>::~unique_ptr() /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:397:6
#45 0x000000000274cc42 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) /root/llvm-build/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:379:3
#46 0x000000000218d9f6 __gnu_cxx::__normal_iterator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>*, std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>>::__normal_iterator(std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>* const&) /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/stl_iterator.h:1068:20
#47 0x000000000218d9f6 std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>::begin() /usr/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/stl_vector.h:871:16
#48 0x000000000218d9f6 void clang::finalize<std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>>(std::vector<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>, std::allocator<std::unique_ptr<clang::TemplateInstantiationCallback, std::default_delete<clang::TemplateInstantiationCallback>>>>&, clang::Sema const&) /root/llvm-build/llvm-project/clang/include/clang/Sema/TemplateInstCallback.h:54:16
#49 0x000000000218d9f6 clang::ParseAST(clang::Sema&, bool, bool) /root/llvm-build/llvm-project/clang/lib/Parse/ParseAST.cpp:183:3
#50 0x0000000002fbdc6c clang::FrontendAction::Execute() /root/llvm-build/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1073:10
#51 0x0000000002fbd771 llvm::Error::getPtr() const /root/llvm-build/llvm-project/llvm/include/llvm/Support/Error.h:279:12
#52 0x0000000002fbd771 llvm::Error::operator bool() /root/llvm-build/llvm-project/llvm/include/llvm/Support/Error.h:239:16
#53 0x0000000002fbd771 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /root/llvm-build/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:1057:23
#54 0x0000000002fbd5c8 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /root/llvm-build/llvm-project/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:272:25
#55 0x0000000002cfef2f cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /root/llvm-build/llvm-project/clang/tools/driver/cc1_main.cpp:294:15
#56 0x0000000002cf879a llvm::SmallVectorBase<unsigned int>::size() const /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallVector.h:91:32
#57 0x0000000002cf879a llvm::SmallVectorTemplateCommon<void*, void>::end() /root/llvm-build/llvm-project/llvm/include/llvm/ADT/SmallVector.h:282:37
#58 0x0000000002cf879a llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::~BumpPtrAllocatorImpl() /root/llvm-build/llvm-project/llvm/include/llvm/Support/Allocator.h:98:42
#59 0x0000000002cf879a ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) /root/llvm-build/llvm-project/clang/tools/driver/driver.cpp:375:1
#60 0x0000000002cfa1b7 clang_main(int, char**, llvm::ToolContext const&) /root/llvm-build/llvm-project/clang/tools/driver/driver.cpp:0:12
#61 0x0000000003038e84 main /root/llvm-build/stage2-prof-use/tools/clang/tools/driver/clang-driver.cpp:17:10
#62 0x00007f47160b9083 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24083)
#63 0x00000000032e771e _start (/root/llvm-build/stage2-prof-use/bin/clang+++0x32e771e)
Segmentation fault

locale-8f8074.cpp

template <bool, class a> using b = a;
struct c {
  c(int, int);
};
template <class> struct B;
template <class d, class = B<d>, class = d> struct g {
  c e;
  template <class f> g(f h, f l) : e(int(), int()) { i(h, l); }
  template <class j, b<1, int> = 0> constexpr void i(j, j);
};
wchar_t *k;
template <> struct B<wchar_t> {
  static void assign(wchar_t &h, wchar_t l) { h = l; }
};
template <class d, class m, class n>
template <class j, int>
constexpr void g<d, m, n>::i(j h, j l) {
  for (; h != l; ++h)
    m::assign(*k, *h);
}
template <class> struct o {
  wchar_t p;
  const wchar_t *q;
  void r() const { g<wchar_t>(&p, q); }
};
template class o<wchar_t>;

cmake command used to build crashing compiler

cmake -G Ninja /root/llvm-build/llvm-project/llvm \
    -DCLANG_DEFAULT_LINKER="lld" \
    -DCLANG_DEFAULT_PIE_ON_LINUX=OFF \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_C_COMPILER_LAUNCHER="ccache" \
    -DCMAKE_C_COMPILER=/root/llvm-build/stage1/install/bin/clang \
    -DCMAKE_C_FLAGS="-fcf-protection=none -fno-semantic-interposition -fno-stack-protector -fstrict-vtable-pointers -fveclib=libmvec -gline-tables-only -march=x86-64-v3 -mharden-sls=none -O3 -Wno-backend-plugin -g -fno-pie" \
    -DCMAKE_CXX_COMPILER_LAUNCHER="ccache" \
    -DCMAKE_CXX_COMPILER=/root/llvm-build/stage1/install/bin/clang++ \
    -DCMAKE_CXX_FLAGS="-fcf-protection=none -fno-semantic-interposition -fno-stack-protector -fstrict-vtable-pointers -fveclib=libmvec -gline-tables-only -march=x86-64-v3 -mharden-sls=none -O3 -Wno-backend-plugin -g -fno-pie" \
    -DCMAKE_EXE_LINKER_FLAGS="-Wl,--push-state,/root/llvm-build/mimalloc/out/release/libmimalloc.a,-lpthread,--pop-state -Bsymbolic -Wl,--emit-relocs -Wl,--gc-sections -Wl,--icf=all -Wl,--lto-O3 -Wl,--lto-whole-program-visibility -Wl,-O3 -Wl,-z,norelro -Wl,-z,now -no-pie -fuse-ld=lld" \
    -DCMAKE_INSTALL_PREFIX="/root/llvm-build/stage2-prof-use-lto/install" \
    -DCMAKE_SHARED_LINKER_FLAGS="-Bsymbolic -Wl,--emit-relocs -Wl,--gc-sections -Wl,--icf=all -Wl,--lto-O3 -Wl,--lto-whole-program-visibility -Wl,-O3 -Wl,-z,norelro -Wl,-z,now -fuse-ld=lld" \
    -DLLVM_BINUTILS_INCDIR="/root/llvm-build/binutils/include" \
    -DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-pc-linux-gnu" \
    -DLLVM_ENABLE_BINDINGS=OFF \
    -DLLVM_ENABLE_PROJECTS='bolt;clang;clang-tools-extra;lld;polly' \
    -DLLVM_ENABLE_RUNTIMES='compiler-rt;libc;libcxxabi;libcxx;libunwind' \
    -DLLVM_ENABLE_ZLIB=FORCE_ON \
    -DLLVM_ENABLE_ZSTD=FORCE_ON \
    -DLLVM_INCLUDE_BENCHMARKS=OFF \
    -DLLVM_INCLUDE_EXAMPLES=OFF \
    -DLLVM_INCLUDE_TESTS=OFF \
    -DLLVM_PROFDATA_FILE="/root/llvm-build/stage2-prof-gen/profiles/clang.profdata" \
    -DLLVM_TARGETS_TO_BUILD=X86 \
    -DLLVM_USE_LINKER="/root/llvm-build/stage1/install/bin/ld.lld"

The crash does not reproduce when Clang is built without PGO. I can provide any additional files if needed.

I'll try to get rid of as much cmake arguments as possible next.

fhahn commented 4 months ago

Interesting. Does this also reproduce when using a different stage1 compiler (e.g. previous LLVM release).

As a first step, we need to clarify if this is caused by a miscompile by the stage1 compiler or if there's some kind of bug in the code that's crashing.

vient commented 4 months ago

Tried to just use clang-17 in last cmake, got unsupported instrumentation profile format version, fair.

Rebuilding instrumented clang-18 using clang-17 now. Funny thing is that I got segmentation fault in lld-17, which I traced to option --icf=all. --icf=safe also crashes, so continuing without it, I think it should not matter here.

Edit: ok i'll first reduce original reproducer, right now it has a ton of options.

vient commented 4 months ago

I've reduced original repro to "gather some profiles", and then

cmake -G Ninja /root/llvm-build/llvm-project/llvm \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER=clang-18 \
    -DCMAKE_CXX_COMPILER=clang++-18 \
    -DCMAKE_C_FLAGS="-fstrict-vtable-pointers -O0 -Wno-backend-plugin -g" \
    -DCMAKE_CXX_FLAGS="-fstrict-vtable-pointers -O0 -Wno-backend-plugin -g" \
    -DCMAKE_INSTALL_PREFIX="/root/llvm-build/stage2-prof-use/install" \
    -DLLVM_PROFDATA_FILE="/root/llvm-build/stage2-prof-gen/profiles/clang.profdata" \
    -DLLVM_USE_LINKER=lld \
    -DLLVM_ENABLE_PROJECTS='clang' \
    -DLLVM_ENABLE_RUNTIMES='compiler-rt;libcxxabi;libcxx;libunwind' \
    -DLLVM_ENABLE_ZLIB=FORCE_ON \
    -DLLVM_ENABLE_ZSTD=FORCE_ON \
    -DLLVM_ENABLE_BINDINGS=OFF \
    -DLLVM_INCLUDE_BENCHMARKS=OFF \
    -DLLVM_INCLUDE_EXAMPLES=OFF \
    -DLLVM_INCLUDE_TESTS=OFF \
    -DLLVM_TARGETS_TO_BUILD=X86
ninja install

clang-18 is taken from LLVM apt repo for Ubuntu 20.04

$ clang-18 --version
Ubuntu clang version 18.1.4 (++20240416114258+e3c832b37b0a-1~exp1~20240416234314.99)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Removing -fstrict-vtable-pointers fixes the issue. Strangely, my stage1 is compiled with -fstrict-vtable-pointers too but it works fine. Seems there is some incompatibility between it and PGO? Also, why does it trigger if -O0 is set?

vient commented 4 months ago

Until now I used profile created from 11k .profraw files, so I decided to try to reduce quantity of raw files. During this process I found a single .profraw which, if included to profile, makes PGO Clang crash.

First I bisected number of profraw files, taking a prefix from sorted list. That process gave me number 1191 - Clang compiled with profile created from first 1190 profraw files does not crash. Then I created profile from that single profraw number 1191, and it crashed. 1 bad profraw out of 1200 seems pretty rare, thought I did not check remaining 9k profiles.

Attaching both profraw file and generated profdata from it 91041_profdata.tar.gz. Everything is done using llvm-18 from official repo. With this, reproducer looks like this

ROOT=/root/llvm-build
mkdir -p "${ROOT}" && cd "${ROOT}"
git clone -b llvmorg-18.1.5 --depth=1 https://github.com/llvm/llvm-project.git
mkdir -p build && cd build
# llvm-profdata-18 merge -output=bad.profdata default_4023465489017712588_0.profraw
cmake -G Ninja "${ROOT}/llvm-project/llvm" \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER=clang-18 \
    -DCMAKE_CXX_COMPILER=clang++-18 \
    -DCMAKE_C_FLAGS="-fstrict-vtable-pointers -O0 -Wno-backend-plugin -g" \
    -DCMAKE_CXX_FLAGS="-fstrict-vtable-pointers -O0 -Wno-backend-plugin -g" \
    -DCMAKE_INSTALL_PREFIX="${PWD}/install" \
    -DLLVM_PROFDATA_FILE="bad.profdata" \
    -DLLVM_USE_LINKER=lld \
    -DLLVM_ENABLE_PROJECTS='clang' \
    -DLLVM_ENABLE_RUNTIMES='compiler-rt;libcxxabi;libcxx;libunwind' \
    -DLLVM_ENABLE_ZLIB=FORCE_ON \
    -DLLVM_ENABLE_ZSTD=FORCE_ON \
    -DLLVM_ENABLE_BINDINGS=OFF \
    -DLLVM_INCLUDE_BENCHMARKS=OFF \
    -DLLVM_INCLUDE_EXAMPLES=OFF \
    -DLLVM_INCLUDE_TESTS=OFF \
    -DLLVM_TARGETS_TO_BUILD=X86
ninja install  # clang segfaults while compiling runtimes

I noticed two kinds of stacktraces, they are pretty similar but still:

 #6 0x0000561a8a8defd4 llvm::PHINode::getBasicBlockIndex(llvm::BasicBlock const*) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Instructions.h:2898:28
 #7 0x0000561a8a8defd4 llvm::PHINode::getIncomingValueForBlock(llvm::BasicBlock const*) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Instructions.h:2904:15
 #8 0x0000561a8a8defd4 llvm::LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(llvm::VFRange&) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8762:18
 #9 0x0000561a8a8b9ec1 std::__uniq_ptr_impl<llvm::VPlan, std::default_delete<llvm::VPlan>>::_M_ptr() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:193:51
#10 0x0000561a8a8b9ec1 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::get() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:464:21
#11 0x0000561a8a8b9ec1 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::operator bool() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:481:16
#12 0x0000561a8a8b9ec1 llvm::LoopVectorizationPlanner::buildVPlansWithVPRecipes(llvm::ElementCount, llvm::ElementCount) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8600:14
#13 0x0000561a8a8b960d llvm::LoopVectorizationPlanner::plan(llvm::ElementCount, unsigned int) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:7419:3
#14 0x0000561a8a8b0b58 llvm::LoopVectorizePass::processLoop(llvm::Loop*) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10004:12

 #6 0x00005567bb2d7fed llvm::Use::get() const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Use.h:66:31
 #7 0x00005567bb2d7fed llvm::PHINode::getOperand(unsigned int) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Instructions.h:2956:1
 #8 0x00005567bb2d7fed llvm::PHINode::getIncomingValue(unsigned int) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Instructions.h:2804:12
 #9 0x00005567bb2d7fed llvm::PHINode::getIncomingValueForBlock(llvm::BasicBlock const*) const /root/llvm-build/llvm-project/llvm/include/llvm/IR/Instructions.h:2906:12
#10 0x00005567bb2d7fed llvm::LoopVectorizationPlanner::tryToBuildVPlanWithVPRecipes(llvm::VFRange&) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8762:18
#11 0x00005567bb2b2ec1 std::__uniq_ptr_impl<llvm::VPlan, std::default_delete<llvm::VPlan>>::_M_ptr() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:193:51
#12 0x00005567bb2b2ec1 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::get() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:464:21
#13 0x00005567bb2b2ec1 std::unique_ptr<llvm::VPlan, std::default_delete<llvm::VPlan>>::operator bool() const /usr/bin/../lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/unique_ptr.h:481:16
#14 0x00005567bb2b2ec1 llvm::LoopVectorizationPlanner::buildVPlansWithVPRecipes(llvm::ElementCount, llvm::ElementCount) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:8600:14
#15 0x00005567bb2b260d llvm::LoopVectorizationPlanner::plan(llvm::ElementCount, unsigned int) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:7419:3
#16 0x00005567bb2a9b58 llvm::LoopVectorizePass::processLoop(llvm::Loop*) /root/llvm-build/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10004:12

These stack traces should be more accurate than in initial post since I don't use any optimizations now.

I also found another profraw file with exactly the same size which does not trigger the crash. Although now I understand that profraw size depends on counter values. Can be there any differences in profraw files besides counter values?

Please let me know if I can do anything else here. Right now I'm out of ideas as someone who is not an LLVM developer.

Naville commented 1 month ago

+1, encountering the same issue with LLVM17.0.2 on Windows, using ClangCL as Stage1, shipped with MSVC 2022

Naville commented 1 month ago

fyi @fhahn and CC @akyrtzi In our test case, the crash was in llvm_blake3_compress_xof_sse41. Disabling with -DLLVM_DISABLE_ASSEMBLY_FILES=ON fixes the issue under PGO. We also previously encountered similar crash when cross-compiling Windows Toolchain under Linux host, without PGO.

I suspect that part of the code is very broken

(Maybe also https://github.com/llvm/llvm-project/issues/81967 )