Closed JustArchi closed 1 month ago
Tagging subscribers to this area: @agocke, @vitek-karas, @vsadov See info in area-owners.md if you want to be subscribed.
Thanks for the report!
Here is the stacktrace with symbols:
(lldb) clrstack -f
OS Thread Id: 0xd6a (1)
Child SP IP Call Site
0000FFFFFFFFCA00 0000AAAAAAEF6C30 ArchiSteamFarm!CanJITOptimizeTLSAccess() + 12 at /__w/1/s/src/coreclr/vm/threadstatics.cpp:813
0000FFFFFFFFCA10 0000AAAAAAE7C98C ArchiSteamFarm!CEEInfo::getFieldInfo(CORINFO_RESOLVED_TOKEN*, CORINFO_METHOD_STRUCT_*, CORINFO_ACCESS_FLAGS, CORINFO_FIELD_INFO*) + 948 at /__w/1/s/src/coreclr/vm/jitinterface.cpp:1438
0000FFFFFFFFCAE0 0000AAAAAB277F14 ArchiSteamFarm!Compiler::impImportBlockCode(BasicBlock*) + 6616 at /__w/1/s/src/coreclr/jit/importer.cpp:9191
0000FFFFFFFFCAE0 0000AAAAAB277EF4 ArchiSteamFarm!Compiler::impImportBlockCode(BasicBlock*) + 6584 at /__w/1/s/src/coreclr/jit/ee_il_dll.hpp:42
0000FFFFFFFFD000 0000AAAAAB2820F4 ArchiSteamFarm!Compiler::impImportBlock(BasicBlock*) + 156 at /__w/1/s/src/coreclr/jit/compiler.hpp:4051
0000FFFFFFFFD0A0 0000AAAAAB284434 ArchiSteamFarm!Compiler::impImport() + 1256 at /__w/1/s/src/coreclr/jit/compiler.hpp:4051
0000FFFFFFFFD0F0 0000AAAAAB2597F0 ArchiSteamFarm!Compiler::fgImport() + 24 at /__w/1/s/src/coreclr/jit/flowgraph.cpp:549
0000FFFFFFFFD120 0000AAAAAB3BC30C ArchiSteamFarm!Phase::Run() + 44 at /__w/1/s/src/coreclr/jit/phase.cpp:62
0000FFFFFFFFD140 0000AAAAAB2190BC ArchiSteamFarm!Compiler::compCompile(void**, unsigned int*, JitFlags*) + 324 at /__w/1/s/src/coreclr/jit/compiler.hpp:4051
0000FFFFFFFFD140 0000AAAAAB2190A0 ArchiSteamFarm!Compiler::compCompile(void**, unsigned int*, JitFlags*) + 296 at /__w/1/s/src/coreclr/jit/phase.h:124
0000FFFFFFFFD4C0 0000AAAAAB21BB4C ArchiSteamFarm!Compiler::compCompileHelper(CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*) + 1328 at /__w/1/s/src/coreclr/jit/compiler.hpp:4051
0000FFFFFFFFD570 0000AAAAAB21B224 ArchiSteamFarm!Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*) + 1172 at /__w/1/s/src/coreclr/jit/compiler.hpp:4051
0000FFFFFFFFD570 0000AAAAAB21B20C ArchiSteamFarm!Compiler::compCompile(CORINFO_MODULE_STRUCT_*, void**, unsigned int*, JitFlags*) + 1148 at /__w/1/s/src/coreclr/jit/compiler.cpp:6545
0000FFFFFFFFD5E0 0000AAAAAB21C5CC ArchiSteamFarm!jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*) + 432 at /__w/1/s/src/coreclr/jit/compiler.cpp:8052
0000FFFFFFFFD5E0 0000AAAAAB21C524 ArchiSteamFarm!jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*) + 264 at /__w/1/s/src/coreclr/jit/compiler.cpp:8007
0000FFFFFFFFD5E0 0000AAAAAB21C51C ArchiSteamFarm!jitNativeCode(CORINFO_METHOD_STRUCT_*, CORINFO_MODULE_STRUCT_*, ICorJitInfo*, CORINFO_METHOD_INFO*, void**, unsigned int*, JitFlags*, void*) + 256 at /__w/1/s/src/coreclr/jit/compiler.cpp:8007
0000FFFFFFFFD760 0000AAAAAB212CDC ArchiSteamFarm!CILJit::compileMethod(ICorJitInfo*, CORINFO_METHOD_INFO*, unsigned int, unsigned char**, unsigned int*) + 136 at /__w/1/s/src/coreclr/jit/ee_il_dll.cpp:294
0000FFFFFFFFD7E0 0000AAAAAAE90DB8 ArchiSteamFarm!invokeCompileMethodHelper(EEJitManager*, CEEInfo*, CORINFO_METHOD_INFO*, CORJIT_FLAGS, unsigned char**, unsigned int*) + 236 at /__w/1/s/src/coreclr/vm/jitinterface.cpp:12512
0000FFFFFFFFD850 0000AAAAAAE90F58 ArchiSteamFarm!invokeCompileMethod(EEJitManager*, CEEInfo*, CORINFO_METHOD_INFO*, CORJIT_FLAGS, unsigned char**, unsigned int*) + 140 at /__w/1/s/src/coreclr/vm/jitinterface.cpp:0
0000FFFFFFFFD8B0 0000AAAAAAE91914 ArchiSteamFarm!UnsafeJitFunction(PrepareCodeConfig*, COR_ILMETHOD_DECODER*, CORJIT_FLAGS*, unsigned int*) + 1856 at /__w/1/s/src/coreclr/vm/jitinterface.cpp:0
0000FFFFFFFFDC80 0000AAAAAAEC9860 ArchiSteamFarm!MethodDesc::JitCompileCodeLocked(PrepareCodeConfig*, COR_ILMETHOD_DECODER*, ListLockEntryBase<NativeCodeVersion>*, unsigned int*) + 220 at /__w/1/s/src/coreclr/vm/threads.inl:198
0000FFFFFFFFDD50 0000AAAAAAEC924C ArchiSteamFarm!MethodDesc::JitCompileCodeLockedEventWrapper(PrepareCodeConfig*, ListLockEntryBase<NativeCodeVersion>*) + 1076 at /__w/1/s/src/coreclr/vm/prestub.cpp:818
0000FFFFFFFFDE90 0000AAAAAAEC8904 ArchiSteamFarm!MethodDesc::JitCompileCode(PrepareCodeConfig*) + 484 at /__w/1/s/src/coreclr/vm/prestub.cpp:0
0000FFFFFFFFDF50 0000AAAAAAEC8460 ArchiSteamFarm!MethodDesc::PrepareILBasedCode(PrepareCodeConfig*) + 920 at /__w/1/s/src/coreclr/vm/prestub.cpp:0
0000FFFFFFFFDFF0 0000AAAAAAE3F254 ArchiSteamFarm!CodeVersionManager::PublishVersionableCodeIfNecessary(MethodDesc*, CallerGCMode, bool*, bool*) + 764 at /__w/1/s/src/coreclr/vm/method.hpp:1994
0000FFFFFFFFE170 0000AAAAAAECCD78 ArchiSteamFarm!MethodDesc::DoPrestub(MethodTable*, CallerGCMode) + 244 at /__w/1/s/src/coreclr/vm/prestub.cpp:2871
0000FFFFFFFFE220 0000AAAAAAECC8B8 ArchiSteamFarm!PreStubWorker + 556 at /__w/1/s/src/coreclr/inc/volatile.h:382
0000FFFFFFFFE268 [PrestubMethodFrame: 0000ffffffffe268] System.Private.CoreLib.dll!System.Threading.Thread.get_CurrentThread()
0000FFFFFFFFE300 0000AAAAAB0BBDF4 ArchiSteamFarm!ThePreStub + 80 at /__w/1/s/src/coreclr/vm/arm64/asmhelpers.S:167
0000FFFFFFFFE430 0000FFFFB79543C8 System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[ArchiSteamFarm.Program+<Main>d__54, ArchiSteamFarm]](<Main>d__54 ByRef) + 96
0000FFFFFFFFE4B0 0000FFFFB7954348 System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Int32, System.Private.CoreLib]].Start[[ArchiSteamFarm.Program+<Main>d__54, ArchiSteamFarm]](<Main>d__54 ByRef) + 40
0000FFFFFFFFE4D0 0000FFFFB79542AC ArchiSteamFarm.dll!ArchiSteamFarm.Program.Main(System.String[]) + 100
0000FFFFFFFFE520 0000FFFFB79541F0 ArchiSteamFarm.dll!ArchiSteamFarm.Program.<Main>(System.String[]) + 40
FFFFFFFFFFFFFFFF 0000FFFFB7954348
FFFFFFFFFFFFFFFF 0000FFFFB79542AC
FFFFFFFFFFFFFFFF 0000FFFFB79541F0
FFFFFFFFFFFFFFFF 0000AAAAAB0BC788 ArchiSteamFarm!CallDescrWorkerInternal + 132 at /__w/1/s/src/coreclr/vm/arm64/calldescrworkerarm64.S:74
0000FFFFFFFFE560 0000AAAAAAF23A2C ArchiSteamFarm!MethodDescCallSite::CallTargetWorker(unsigned long const*, unsigned long*, int) + 768 at /__w/1/s/src/coreclr/pal/inc/pal.h:4237
0000FFFFFFFFE560 0000AAAAAAF239D8 ArchiSteamFarm!MethodDescCallSite::CallTargetWorker(unsigned long const*, unsigned long*, int) + 684 at /__w/1/s/src/coreclr/vm/threads.inl:42
0000FFFFFFFFE7C0 0000AAAAAAE1A384 ArchiSteamFarm!RunMain(MethodDesc*, short, int*, PtrArray**) + 756 at /__w/1/s/src/coreclr/vm/callhelpers.h:465
0000FFFFFFFFE7C0 0000AAAAAAE1A370 ArchiSteamFarm!RunMain(MethodDesc*, short, int*, PtrArray**) + 736 at /__w/1/s/src/coreclr/vm/callhelpers.h:465
0000FFFFFFFFE7C0 0000AAAAAAE1A238 ArchiSteamFarm!RunMain(MethodDesc*, short, int*, PtrArray**) + 424 at /__w/1/s/src/coreclr/vm/callhelpers.h:203
0000FFFFFFFFE7C0 0000AAAAAAE1A238 ArchiSteamFarm!RunMain(MethodDesc*, short, int*, PtrArray**) + 424 at /__w/1/s/src/coreclr/vm/callhelpers.h:203
0000FFFFFFFFE7C0 0000AAAAAAE1A1F4 ArchiSteamFarm!RunMain(MethodDesc*, short, int*, PtrArray**) + 356 at /__w/1/s/src/coreclr/vm/threads.inl:37
0000FFFFFFFFE9D0 0000AAAAAAE1A6DC ArchiSteamFarm!Assembly::ExecuteMainMethod(PtrArray**, int) + 272 at /__w/1/s/src/coreclr/vm/assembly.cpp:0
0000FFFFFFFFEC60 0000AAAAAAE40F14 ArchiSteamFarm!CorHost2::ExecuteAssembly(unsigned int, char16_t const*, int, char16_t const**, unsigned int*) + 652 at /__w/1/s/src/coreclr/vm/corhost.cpp:350
0000FFFFFFFFED80 0000AAAAAB117CF4 ArchiSteamFarm!coreclr_execute_assembly + 252 at /__w/1/s/src/coreclr/dlls/mscoree/exports.cpp:0
0000FFFFFFFFEE00 0000AAAAAADF69B4 ArchiSteamFarm!run_app_for_context(hostpolicy_context_t const&, int, char const**) + 856 at /__w/1/s/src/native/corehost/hostpolicy/hostpolicy.cpp:250
0000FFFFFFFFEED0 0000AAAAAADF7928 ArchiSteamFarm!corehost_main + 340 at /__w/1/s/src/native/corehost/hostpolicy/hostpolicy.cpp:0
0000FFFFFFFFEED0 0000AAAAAADF7908 ArchiSteamFarm!corehost_main + 308 at /__w/1/s/src/native/corehost/hostpolicy/hostpolicy.cpp:281
0000FFFFFFFFEF90 0000AAAAAADCE608 ArchiSteamFarm!fx_muxer_t::handle_exec_host_command(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, host_startup_info_t const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::unordered_map<known_options, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>, known_options_hash, std::equal_to<known_options>, std::allocator<std::pair<known_options const, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>>>> const&, int, char const**, int, host_mode_t, bool, char*, int, int*) + 1188 at /__w/1/s/src/native/corehost/fxr/fx_muxer.cpp:0
0000FFFFFFFFEF90 0000AAAAAADCE424 ArchiSteamFarm!fx_muxer_t::handle_exec_host_command(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, host_startup_info_t const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::unordered_map<known_options, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>, known_options_hash, std::equal_to<known_options>, std::allocator<std::pair<known_options const, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>>>> const&, int, char const**, int, host_mode_t, bool, char*, int, int*) + 704 at /crossrootfs/arm64/usr/include/aarch64-linux-gnu/c++/5/bits/gthr-default.h:747
0000FFFFFFFFEF90 0000AAAAAADCE34C ArchiSteamFarm!fx_muxer_t::handle_exec_host_command(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, host_startup_info_t const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::unordered_map<known_options, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>, known_options_hash, std::equal_to<known_options>, std::allocator<std::pair<known_options const, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>>>> const&, int, char const**, int, host_mode_t, bool, char*, int, int*) + 488 at /crossrootfs/arm64/usr/include/c++/5/bits/basic_string.h:716
0000FFFFFFFFF0D0 0000AAAAAADCD92C ArchiSteamFarm!fx_muxer_t::execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, int, char const**, host_startup_info_t const&, char*, int, int*) + 760 at /__w/1/s/src/native/corehost/fxr/fx_muxer.cpp:0
0000FFFFFFFFF200 0000AAAAAADC7FA8 ArchiSteamFarm!hostfxr_main_bundle_startupinfo + 256 at /__w/1/s/src/native/corehost/fxr/hostfxr.cpp:48
0000FFFFFFFFF2E0 0000AAAAAADC3890 ArchiSteamFarm!exe_start(int, char const**) + 732 at /__w/1/s/src/native/corehost/hostmisc/utils.h:160
0000FFFFFFFFF460 0000AAAAAADC3C00 ArchiSteamFarm!main + 184 at /__w/1/s/src/native/corehost/corehost.cpp:321
0000FFFFFFFFF4D0 0000FFFFF79B84C4 libc.so.6!___lldb_unnamed_symbol3097 + 116
0000FFFFFFFFF5E0 0000FFFFF79B8598 libc.so.6!__libc_start_main + 152
0000FFFFFFFFF640 0000AAAAAADC1FB4 ArchiSteamFarm!_start + 52
Probably this will fix it https://github.com/dotnet/runtime/compare/main...am11:runtime:patch-10?w=1 (borrowed condition from amd64 branch below). @davidwrighton, thoughts?
@am11, your patch modifies the TARGET_LOONGARCH64
variant of this code. However, I agree, something seems quite off here. @kunalspathak please take a look.
@am11, your patch modifies the
TARGET_LOONGARCH64
variant of this code. However, I agree, something seems quite off here. @kunalspathak please take a look.
The nullptr
check won't help here, because the TLSResolver is returning 0x200
which will bypass the nullptr
check, but still fail.
which is different than the pattern seen in helloworld app on the same machine.
I am still investigating scenarios when this can happen.
Alright, I was able to repro this on a "hello world" project. Basically, if an app is built with -p:PublishSingleFile=true -p:PublishTrimmed=true --self-contained
, we will hit this. It doesn't even need to have a single ThreadStatic
in them. Looking for options...
Alright, I was able to repro this on a "hello world" project. Basically, if an app is built with -p:PublishSingleFile=true -p:PublishTrimmed=true --self-contained
@agocke, @vitek-karas, I remember we moved some PublishSingleFile tests from runtime to installer repo because they were slowing down the CI. Is it so that those tests are not running after sdk <- installer repos merging? "hello world" being broken in that configuration seems to be a major test gap. (or maybe we don't have linux-arm64 test leg in sdk?)
I don't know if it's still the case, but couple years back SDK CI didn't run on arm64 at all, and we relied on installer to provide that coverage. If it's still the case after the merge of the repos, that's a big bummer. I agree this is a test hole, but I don't know where is the best place to cover it. Originally it was really difficult to run single-file tests in runtime repo and thus we relied on SDK. Maybe it's better now - I know @elinor-fung did lot of work to simplify the testing infra around this.
I agree this is a test hole, but I don't know where is the best place to cover it. Originally it was really difficult to run single-file tests in runtime repo and thus we relied on SDK. Maybe it's better now - I know @elinor-fung did lot of work to simplify the testing infra around this.
We added such capability in #42972 together with regular CI runs but then it got pulled to cut CI costs in #65453.
I think running the full test suite in every configuration does seem expensive, but having a single realworld-y webapp and publish it in different configurations, just enough to see runtime starts up, should be doable in a regular GitHub Actions (download artifacts, publish, run and collect results). This is not the first time trimmed/r2r/singlefile has failed at startup due to the lack of tests.
I believe this was fixed by https://github.com/dotnet/runtime/pull/108564
Description
Hey, I'm currently updating my ASF project to .NET 9 (rc1), during testing it seems that
linux-arm64
build results in SIGSEGV during running.It's important to note that self-contained publish, trimmed publish as well as single-file publish all work fine. It's a combination of single-file trimmed publish for linux-arm64 that triggers this.
Reproduction Steps
If you want to just run the resulting app, https://github.com/JustArchiNET/ArchiSteamFarm/actions/runs/11056649514/artifacts/1983547331,
unzip
then./ArchiSteamFarm
.If you want to build and reproduce from source, which is probably what you're after:
Expected behavior
Program should run like expected - starting, not crashing, printing stuff on the console.
Actual behavior
Process crashes with SIGSEGV before it manages to do anything productive. I've tried
gdb
but I'm not an expert in this matter, attaching the results anyway below.Regression?
Yes, .NET 8 worked fine in this scenario. Other .NET 9 variants work fine as well in the same settings, which heavily suggest that this is isolated linux-arm64 runtime regression.
Known Workarounds
None, using other variants such as linux-x64 works fine though.
Using only trimming, only single file publishing or nothing at all works as well.
Configuration
Like above, you need
linux-arm64
trimmed single-file build. I've reproduced it on Raspberry Pi 5 running standard Raspberry Pi OS (lite) updated to current Debian testing sources. I doubt it's linked with some specific hardware though, so any other linux-arm64 machine should be sufficient as well.Other information
Discovered while working on https://github.com/JustArchiNET/ArchiSteamFarm/pull/3244#issuecomment-2378682020
Let me know if there is any more information I can provide.