llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.23k stars 12.07k forks source link

LLVM test failure on aarch64-linux #116753

Open RossComputerGuy opened 3 days ago

RossComputerGuy commented 3 days ago

I was updating the LLVM git in Nixpkgs and ran into this failure with commit 3f9d02aae87b7c778b86cb79ebd4b64760653079 on aarch64-linux. LLVM git didn't have this problem a month ago when I last built it but now it does with that commit. This is running on an Ampere Altra Q64-22 running NixOS with a 64k page size.

llvm> [1/2] Preparing lit tests
llvm> [1/2] Running all regression tests
llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using split-file: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/split-file
llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using yaml2obj: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/yaml2obj
llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using llvm-objcopy: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/llvm-objcopy
llvm> An error occurred retrieving the git revision: [Errno 2] No such file or directory: 'git'
llvm> -- Testing: 58253 tests, 32 workers --
llvm> FAIL: LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82 (55605 of 58253)
llvm> ******************** TEST 'LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82' FAILED ********************
llvm> Script(shard):
llvm> --
llvm> GTEST_OUTPUT=json:/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-19985-79-82.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=82 GTEST_SHARD_INDEX=79 /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests
llvm> --
llvm>
llvm> Note: This is test shard 80 of 82.
llvm> [==========] Running 2 tests from 2 test suites.
llvm> [----------] Global test environment set-up.
llvm> [----------] 1 test from ObjectLinkingLayerTest
llvm> [ RUN      ] ObjectLinkingLayerTest.ClaimLateDefinedWeakSymbols
llvm> [       OK ] ObjectLinkingLayerTest.ClaimLateDefinedWeakSymbols (0 ms)
llvm> [----------] 1 test from ObjectLinkingLayerTest (0 ms total)
llvm>
llvm> [----------] 1 test from JITLinkRedirectionManagerTest
llvm> [ RUN      ] JITLinkRedirectionManagerTest.BasicRedirectionOperation
llvm> Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
llvm> 0  libLLVM.so.20.0git 0x0000ffffefaeb708 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72
llvm> 1  libLLVM.so.20.0git 0x0000ffffefae9144
llvm> 2  linux-vdso.so.1    0x0000fffff7fa07e0 __kernel_rt_sigreturn + 0
llvm> 3  libLLVM.so.20.0git 0x0000fffff220c2a0 llvm::orc::InProcessMemoryAccess::writePointersAsync(llvm::ArrayRef<llvm::orc::tpctypes::PointerWrite>, llvm::unique_function<void (llvm::Error)>) + 64
llvm> 4  libLLVM.so.20.0git 0x0000fffff2210d1c
llvm> 5  libLLVM.so.20.0git 0x0000fffff2211e6c llvm::orc::JITLinkRedirectableSymbolManager::redirect(llvm::orc::JITDylib&, llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef>> const&) + 2764
llvm> 6  OrcJITTests        0x0000000000519da0 JITLinkRedirectionManagerTest_BasicRedirectionOperation_Test::TestBody() + 1408
llvm> 7  OrcJITTests        0x0000000000549a58
llvm> 8  OrcJITTests        0x0000000000549e50 testing::TestInfo::Run() + 688
llvm> 9  OrcJITTests        0x000000000054a420
llvm> 10 OrcJITTests        0x0000000000558274 testing::internal::UnitTestImpl::RunAllTests() + 2980
llvm> 11 OrcJITTests        0x0000000000548f10 testing::UnitTest::Run() + 112
llvm> 12 OrcJITTests        0x0000000000459a7c main + 108
llvm> 13 libc.so.6          0x0000ffffee536294
llvm> 14 libc.so.6          0x0000ffffee536368 __libc_start_main + 152
llvm> 15 OrcJITTests        0x000000000045a170 _start + 48
llvm>
llvm> --
llvm> exit: -11
llvm> --
llvm> shard JSON output does not exist: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-19985-79-82.json
llvm> ********************
llvm> ********************
llvm> Failed Tests (1):
llvm>   LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82
llvm>
llvm>
llvm> Testing Time: 189.48s
llvm>
llvm> Total Discovered Tests: 64003
llvm>   Skipped          :    20 (0.03%)
llvm>   Unsupported      :  2699 (4.22%)
llvm>   Passed           : 61106 (95.47%)
llvm>   Expectedly Failed:   177 (0.28%)
llvm>   Failed           :     1 (0.00%)
llvm> FAILED: CMakeFiles/check-all /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/CMakeFiles/check-all
llvm> cd /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build && /nix/store/q52ls2864j1wqxai5g1mhf558y2aj32z-python3-3.12.7-env/bin/python3.12 /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/./bin/llvm-lit -svj32 --no-progress-bar --param polly_site_config=/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test/lit.site.cfg --param polly_unit_site_config=/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test/Unit/lit.site.cfg /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/utils/mlgo-utils /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/utils/lit /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/test
llvm> ninja: build stopped: subcommand failed.
llvmbot commented 2 days ago

@llvm/issue-subscribers-orcjit

Author: Tristan Ross (RossComputerGuy)

I was updating the LLVM git in Nixpkgs and ran into this failure with commit 3f9d02aae87b7c778b86cb79ebd4b64760653079 on aarch64-linux. LLVM git didn't have this problem a month ago when I last built it but now it does with that commit. This is running on an Ampere Altra Q64-22 running NixOS with a 64k page size. ``` llvm> [1/2] Preparing lit tests llvm> [1/2] Running all regression tests llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using split-file: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/split-file llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using yaml2obj: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/yaml2obj llvm> llvm-lit: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/utils/lit/lit/llvm/config.py:506: note: using llvm-objcopy: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/bin/llvm-objcopy llvm> An error occurred retrieving the git revision: [Errno 2] No such file or directory: 'git' llvm> -- Testing: 58253 tests, 32 workers -- llvm> FAIL: LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82 (55605 of 58253) llvm> ******************** TEST 'LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82' FAILED ******************** llvm> Script(shard): llvm> -- llvm> GTEST_OUTPUT=json:/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-19985-79-82.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=82 GTEST_SHARD_INDEX=79 /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests llvm> -- llvm> llvm> Note: This is test shard 80 of 82. llvm> [==========] Running 2 tests from 2 test suites. llvm> [----------] Global test environment set-up. llvm> [----------] 1 test from ObjectLinkingLayerTest llvm> [ RUN ] ObjectLinkingLayerTest.ClaimLateDefinedWeakSymbols llvm> [ OK ] ObjectLinkingLayerTest.ClaimLateDefinedWeakSymbols (0 ms) llvm> [----------] 1 test from ObjectLinkingLayerTest (0 ms total) llvm> llvm> [----------] 1 test from JITLinkRedirectionManagerTest llvm> [ RUN ] JITLinkRedirectionManagerTest.BasicRedirectionOperation llvm> Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): llvm> 0 libLLVM.so.20.0git 0x0000ffffefaeb708 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72 llvm> 1 libLLVM.so.20.0git 0x0000ffffefae9144 llvm> 2 linux-vdso.so.1 0x0000fffff7fa07e0 __kernel_rt_sigreturn + 0 llvm> 3 libLLVM.so.20.0git 0x0000fffff220c2a0 llvm::orc::InProcessMemoryAccess::writePointersAsync(llvm::ArrayRef<llvm::orc::tpctypes::PointerWrite>, llvm::unique_function<void (llvm::Error)>) + 64 llvm> 4 libLLVM.so.20.0git 0x0000fffff2210d1c llvm> 5 libLLVM.so.20.0git 0x0000fffff2211e6c llvm::orc::JITLinkRedirectableSymbolManager::redirect(llvm::orc::JITDylib&, llvm::DenseMap<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr, void>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, llvm::orc::ExecutorSymbolDef>> const&) + 2764 llvm> 6 OrcJITTests 0x0000000000519da0 JITLinkRedirectionManagerTest_BasicRedirectionOperation_Test::TestBody() + 1408 llvm> 7 OrcJITTests 0x0000000000549a58 llvm> 8 OrcJITTests 0x0000000000549e50 testing::TestInfo::Run() + 688 llvm> 9 OrcJITTests 0x000000000054a420 llvm> 10 OrcJITTests 0x0000000000558274 testing::internal::UnitTestImpl::RunAllTests() + 2980 llvm> 11 OrcJITTests 0x0000000000548f10 testing::UnitTest::Run() + 112 llvm> 12 OrcJITTests 0x0000000000459a7c main + 108 llvm> 13 libc.so.6 0x0000ffffee536294 llvm> 14 libc.so.6 0x0000ffffee536368 __libc_start_main + 152 llvm> 15 OrcJITTests 0x000000000045a170 _start + 48 llvm> llvm> -- llvm> exit: -11 llvm> -- llvm> shard JSON output does not exist: /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/unittests/ExecutionEngine/Orc/./OrcJITTests-LLVM-Unit-19985-79-82.json llvm> ******************** llvm> ******************** llvm> Failed Tests (1): llvm> LLVM-Unit :: ExecutionEngine/Orc/./OrcJITTests/79/82 llvm> llvm> llvm> Testing Time: 189.48s llvm> llvm> Total Discovered Tests: 64003 llvm> Skipped : 20 (0.03%) llvm> Unsupported : 2699 (4.22%) llvm> Passed : 61106 (95.47%) llvm> Expectedly Failed: 177 (0.28%) llvm> Failed : 1 (0.00%) llvm> FAILED: CMakeFiles/check-all /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/CMakeFiles/check-all llvm> cd /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build && /nix/store/q52ls2864j1wqxai5g1mhf558y2aj32z-python3-3.12.7-env/bin/python3.12 /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/./bin/llvm-lit -svj32 --no-progress-bar --param polly_site_config=/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test/lit.site.cfg --param polly_unit_site_config=/build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test/Unit/lit.site.cfg /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/utils/mlgo-utils /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/tools/polly/test /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/utils/lit /build/llvm-src-20.0.0-unstable-2024-11-17/llvm/build/test llvm> ninja: build stopped: subcommand failed. ```
lhames commented 2 days ago

The failure looks unrelated to the commit. @RossComputerGuy -- I suspect this test is flaky and may only fail sometimes -- how did you bisect the failure?

lhames commented 2 days ago

More importantly I guess: Does this fail consistently on top-of-tree? If so we can use that to debug the issue.

RossComputerGuy commented 2 days ago

I didn't bisect it, this was the first commit which I saw the failure on. It isn't flaky, it has happened 5 times in a row. I've ran this with 3 different commits since this issue was opened, all HEAD.