search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.03k stars 4.68k forks source link

linux ARM64 dotnet7 regression preview 4 to preview 5, 6 and 7 #72645

Closed RobertHenry6bev closed 2 years ago

RobertHenry6bev commented 2 years ago

Description

When I attempt to build my application, there is an error coming from crossgen2. I get a different error with dotnet7 preview 5 vs preview 6. Both errors smell like SIGSEGV, and both smell of a corrupted heap. Perhaps other dotnet7 previews have this corruption, but it isn't noticed.

/home/msft/dotnet_all/dotnet7.0/sdk/7.0.100-preview.5.22307.18/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(464,5): error : Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException. [/mnt/msft/dotnet_techempower/aspnet/benchmarks/src/BenchmarksApps/Kestrel/PlatformBenchmarks/PlatformBenchmarks.csproj]

Reproduction Steps

Linux jammy on ARM64, using dotnet7 preview 5 or 6 Running under gdb, the top few stack frame looks like: Switching to Thread 0xfffff7ff36a0 (LWP 2813)]

0x0000aaaaaae57160 in ILCompiler_TypeSystem_Internal_NativeFormat_TypeHashingAlgorithms__ComputeNameHashCode ()

(gdb) where

0 0x0000aaaaaae57160 in ILCompiler_TypeSystem_Internal_NativeFormat_TypeHashingAlgorithms__ComputeNameHashCode ()

1 0x0000aaaaaae5b214 in ILCompiler_TypeSystem_Internal_TypeSystem_InstantiatedType__GetMethod ()

2 0x0000aaaaaae757e4 in ILCompiler_TypeSystem_Internal_TypeSystem_Ecma_EcmaModule__ResolveMemberReference ()

Expected behavior

no internal errors

Actual behavior

heap corruption

Regression?

No response

Known Workarounds

No response

Configuration

linux (ubuntu jammy 22.04) arm64 dotnet7 preview 4, 5 and 6 (4 OK[sic]) The error's presence and location seem to move around depending on the number of cores available for the job. 3 cores seemed to tickle this problem.

Other information

No response

dotnet-issue-labeler[bot] commented 2 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

RobertHenry6bev commented 2 years ago

area-crossgen2-coreclr


agocke commented 2 years ago

This might be a NativeAOT bug. @RobertHenry6bev any chance you could get a dump file for the crash?

RobertHenry6bev commented 2 years ago

On ARM64 Linux ubuntu 22.04 with dotnet7 preview5, I set environment variables COMPlus_EnableCrashReport=1 COMPlus_DbgEnableMiniDump=1 COMPlus_CreateDumpDiagnostics=1

I set ulimit -c unlimited I set kernel config /proc/sys/kernel/core_pattern to /tmp/cores/core.%e.%p.%h.%t I ran my build script to build my dotnet world. I got 4 messages like: /home/robhenry/dotnet_all/dotnet7.0/sdk/7.0.100-preview.5.22307.18/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(464,5): error : Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException. [/mnt/robhenry/dotnet_techempower/aspnet/benchmarks/src/BenchmarksApps/Kestrel/PlatformBenchmarks/PlatformBenchmarks.csproj] I got 4 core dump files.

I am unable to attach a zip of the corefile, since that is too large. The zip has been transmitted to agocke out-of-band.

RobertHenry6bev commented 2 years ago

I'm trying to build and run a dotnet runtime using gcc with the -fsanitize=thread (tsan), but I'm getting dynamic linking failures preventing me from making more progress

MichalStrehovsky commented 2 years ago

Didn't realize we have this bug. Copying from #72831 that I'll close as a dup:

Very easy to hit on a Raspberry Pi 4 on AArch64 Ubuntu 22.04. We also have a customer report for hitting this on a beefier Ampere device (there it requires running as /usr/bin/taskset --all-tasks --cpu-list 1,2,3 to limit the number of CPUs). Probably any ARM64 device would hit this.

The repro I just tried on Ubuntu:

curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --version 7.0.100-rc.1.22375.3 --install-dir ~/netrc/
~/netrc/dotnet new webapi -o webapi
cd webapi
cat /dev/tty > nuget.config
<configuration>
  <packageSources>
    <add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
  </packageSources>
</configuration>
^Z
~/netrc/dotnet publish -r linux-arm64 -c Release /p:PublishTrimmed=true /p:PublishReadyToRun=true

The stack traces are all over the place:

error : Error: Object reference not set to an instance of an object. [/home/michal/webapi/webapi.csproj]
error : System.NullReferenceException: Object reference not set to an instance of an object. [/home/michal/webapi/webapi.csproj]
error :    at System.Threading.Lock.ReleaseContended() + 0x50 [/home/michal/webapi/webapi.csproj]
error :    at System.Threading.Lock.Release() + 0x70 [/home/michal/webapi/webapi.csproj]
error :    at System.Collections.Concurrent.ConcurrentDictionary`2.TryAddInternal(TKey, Nullable`1, TValue, Boolean, Boolean, TValue&) + 0x348 [/home/michal/webapi/webapi.csproj]
error :    at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey, Func`2) + 0xbc [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysis.ReadyToRun.CopiedMetadataBlobNode.WriteMethodTableRvas(NodeFactory, ObjectDataBuilder&, BlobReader&) + 0xd0 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysis.ReadyToRun.CopiedMetadataBlobNode.GetData(NodeFactory, Boolean) + 0x1b0 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysis.ObjectNode.GetStaticDependencies(NodeFactory) + 0x48 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependenciesImpl(DependencyNodeCore`1) + 0x48 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependencies(DependencyNodeCore`1) + 0x40 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ProcessMarkStack() + 0xd4 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() + 0x4c [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.ReadyToRunCodegenCompilation.Compile(String) + 0x3c [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.Program.RunSingleCompilation(Dictionary`2, InstructionSetSupport, String, Dictionary`2, HashSet`1, CompilerTypeSystemContext) + 0x1418 [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.Program.Run(String[]) + 0xbbc [/home/michal/webapi/webapi.csproj]
error :    at ILCompiler.Program.Main(String[]) + 0x34 [/home/michal/webapi/webapi.csproj]
error :  ---> ILCompiler.CodeGenerationFailedException: Code generation failed for method '[Microsoft.AspNetCore.Http.Abstractions]Microsoft.AspNetCore.Http.PathString.StartsWithSegments(PathString,StringComparison,PathString&)' [/home/michal/webapi/webapi.csproj]
error :  ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. [/home/michal/webapi/webapi.csproj]
error :    at System.ThrowHelper.ThrowArgumentOutOfRangeException() + 0x28 [/home/michal/webapi/webapi.csproj]
error :    at System.Text.Encoding.GetByteCountWithFallback(Char*, Int32, Int32) + 0x50 [/home/michal/webapi/webapi.csproj]
error :    at System.Text.UTF8Encoding.GetByteCount(String) + 0x84 [/home/michal/webapi/webapi.csproj]
error :    at Internal.JitInterface.CorInfoImpl.StringToUTF8(String) + 0x30 [/home/michal/webapi/webapi.csproj]
error :    at Internal.JitInterface.CorInfoImpl.getMethodNameFromMetadata(CORINFO_METHOD_STRUCT_*, Byte**, Byte**, Byte**) + 0x140 [/home/michal/webapi/webapi.csproj]
error :    at Internal.JitInterface.CorInfoImpl._getMethodNameFromMetadata(IntPtr, IntPtr*, CORINFO_METHOD_STRUCT_*, Byte**, Byte**, Byte**) + 0x5c [/home/michal/webapi/webapi.csproj]

I have a crashdump that I was looking at that has:

[0x3]   crossgen2!SystemNative_Abort + 0x8   
[0x4]   crossgen2!S_P_CoreLib_System_RuntimeExceptionHelpers__FailFast_1 + 0xb8   
[0x5]   crossgen2!S_P_CoreLib_System_RuntimeExceptionHelpers__GetRuntimeException + 0x33c   
[0x6]   crossgen2!S_P_CoreLib_System_Runtime_EH__GetClasslibException + 0x38   
[0x7]   crossgen2!RhThrowHwEx + 0xd8   
[0x8]   crossgen2!RhpThrowHwEx + 0x88   
[0x9]   crossgen2!System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2<System___Canon__Bool>__TryGetValueInternal + 0x44   
[0xa]   crossgen2!System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2<System___Canon__Bool>__GetOrAdd + 0x78   
[0xb]   crossgen2!ILCompiler_ReadyToRun_ILCompiler_DependencyAnalysis_ReadyToRun_ModuleTokenResolver__AddModuleTokenForType + 0x13c   
[0xc]   crossgen2!ILCompiler_ReadyToRun_ILCompiler_DependencyAnalysis_ReadyToRun_ModuleTokenResolver_TokenResolverProvider__GetTypeFromReference + 0x90   
[0xd]   crossgen2!System_Reflection_Metadata_System_Reflection_Metadata_Ecma335_SignatureDecoder_2<System___Canon__System___Canon>__DecodeTypeHandle + 0x124   
[0xe]   crossgen2!System_Reflection_Metadata_System_Reflection_Metadata_Ecma335_SignatureDecoder_2<System___Canon__System___Canon>__DecodeType_0 + 0x348   
[0xf]   crossgen2!System_Reflection_Metadata_System_Reflection_Metadata_Ecma335_SignatureDecoder_2<System___Canon__System___Canon>__DecodeType_0 + 0x2a8   
[0x10]   crossgen2!ILCompiler_ReadyToRun_ILCompiler_DependencyAnalysis_ReadyToRun_ModuleTokenResolver__AddModuleTokenForType + 0x124   
[0x11]   crossgen2!ILCompiler_ReadyToRun_ILCompiler_DependencyAnalysis_ReadyToRun_ModuleTokenResolver__AddModuleTokenForMethod + 0x21c   
[0x12]   crossgen2!ILCompiler_ReadyToRun_Internal_JitInterface_CorInfoImpl__resolveToken + 0x1a8   
[0x13]   crossgen2!ILCompiler_ReadyToRun_Internal_JitInterface_CorInfoImpl___resolveToken + 0x3c

We've seen a bunch of apparent corruptions around ConcurrentDictionary even before the NativeAOT switchover: see #69192. But I'm not able to repro this with crossgen2 without NativeAOT anymore. So it's either a NativeAOT bug or the bug doesn't repro without regions (we still run GC in a different configuration than the rest of the shipping product unfortunately).

EgorBo commented 2 years ago

I hit a gc assert when I tried to enable Frozen Segments for CoreCLR https://github.com/dotnet/runtime/pull/49576#issuecomment-1199995582 (doesn't repro when I disable gc regions)

can it be related since NativeAOT also uses frozen segments?

MichalStrehovsky commented 2 years ago

We couldn't enable regions yet in NativeAOT because of some bug. Possibly the bug you found. So we don't run with region in NativeAOT yet. We have a 7.0 issue to get them running.

EgorBo commented 2 years ago

We couldn't enable regions yet in NativeAOT because of some bug. Possibly the bug you found. So we don't run with region in NativeAOT yet. We have a 7.0 issue to get them running.

Filed https://github.com/dotnet/runtime/pull/73110. There is another bug I've just found - it seems when you register a frozen segment you should not set ibAllocated to the size of the segment you requested (not sure my change is correct though) and update it as you allocate objects on that heap

RobertHenry6bev commented 2 years ago

This issue is still here with yesterday's preview release: 7.0.100-preview.7.22377.5 It is still blocking forward progress.

JulieLeeMSFT commented 2 years ago

@MichalStrehovsky, is your team working on the bug fix for this issue? Who is the owner of this issue? It would be good to have an assignee.

@RobertHenry6bev, as I explained in Teams chat, we try to resolve blocking-release issues before GA release.

MichalStrehovsky commented 2 years ago

@MichalStrehovsky, is your team working on the bug fix for this issue? Who is the owner of this issue? It would be good to have an assignee.

I raised it to Jeff today that we need an owner. There is another crossgen2 crash (without NativeAOT this time) on Windows ARM64: https://github.com/dotnet/runtime/issues/71702. It's not clear if this is codegen, GC, VM, or what, or even whether they are related. I looked at a crashdump - it is a corruption.

JulieLeeMSFT commented 2 years ago

cc @mangod9.

mangod9 commented 2 years ago

It is still blocking forward progress.

@RobertHenry6bev just checking whether you are able to get it to publish from an x64 host, as in you can always publish for lin-arm64 from a x64 sdk (from any OS)?

RobertHenry6bev commented 2 years ago

I have not tried this suggestion. Given the kinds and locations of machines we run this workload on, everything needs to work out of the box.

On Thu, Aug 11, 2022 at 4:41 PM Manish Godse @.***> wrote:

It is still blocking forward progress.

@RobertHenry6bev https://github.com/RobertHenry6bev just checking whether you are able to get it to publish from an x64 host, as in you can always publish for lin-arm64 from a x64 sdk (from any OS)?

— Reply to this email directly, view it on GitHub https://github.com/dotnet/runtime/issues/72645#issuecomment-1212595184, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBLLY6PCAU7KN2FORS6UV3VYWFT3ANCNFSM54JESG7Q . You are receiving this because you were mentioned.Message ID: @.***>

mangod9 commented 2 years ago

I am able to repro it too, this stack consistently seems to be showing up on recent builds:


    frame #7: 0x0000aad846a37c44 crossgen2`RhpThrowHwEx at ExceptionHandling.S:246
    frame #8: 0x0000aad846c42f34 crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_InstantiatedType__GetMethod(this=0x0000ff0e5afb14d8, name=<unavailable>, signature=<unavailable>, substitution=<unavailable>) at InstantiatedType.cs:161
    frame #9: 0x0000aad846c42f34 crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_InstantiatedType__GetMethod(this=0x0000000000000000, name=<unavailable>, signature=<unavailable>, substitution=<unavailable>) at InstantiatedType.cs:161
    frame #10: 0x0000aad846c5a878 crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_Ecma_EcmaModule__ResolveMemberReference(this=<unavailable>, handle=<unavailable>) at EcmaModule.cs:483
    frame #11: 0x0000aad846c650dc crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_Ecma_EcmaModule_EcmaObjectLookupHashtable__CreateValueFromKey(this=<unavailable>, handle=<unavailable>) at EcmaModule.cs:118
    frame #12: 0x0000aad846d9e0d0 crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_LockFreeReaderHashtable_2<System_Reflection_Metadata_System_Reflection_Metadata_EntityHandle__System___Canon>__CreateValueAndEnsureValueIsInTable(this=0x0000ff0e5af64dd0, key=<unavailable>) at LockFreeReaderHashtable.cs:559
    frame #13: 0x0000aad846c5a12c crossgen2`ILCompiler_TypeSystem_Internal_TypeSystem_Ecma_EcmaModule__GetObject(this=<unavailable>, handle=<unavailable>, notFoundBehavior=Throw) at EcmaModule.cs:378
    frame #14: 0x0000aad846cc8608 crossgen2`ILCompiler_ReadyToRun_Internal_JitInterface_CorInfoImpl__ResolveTokenInScope(methodIL=<unavailable>, typeOrMethodContext=<unavailable>, token=<unavailable>) at CorInfoImpl.cs:1585
    frame #15: 0x0000aad846cc89e4 crossgen2`ILCompiler_ReadyToRun_Internal_JitInterface_CorInfoImpl__resolveToken(this=0x0000ff0d5afa9e60, pResolvedToken=0x0000ff0c3eb5bc80) at CorInfoImpl.cs:1656
    frame #16: 0x0000aad846cbe5ac crossgen2`ILCompiler_ReadyToRun_Internal_JitInterface_CorInfoImpl___resolveToken(thisHandle=<unavailable>, ppException=<unavailable>, pResolvedToken=<unavailable>) at CorInfoBase.cs:445```
kunalspathak commented 2 years ago

I am able to repro it too, this stack consistently seems to be showing up on recent builds:

@mangod9 - can you confirm the steps you used to repro this? And if I understand correctly, you were able to repro it on linux/arm64?

mangod9 commented 2 years ago

I followed the steps from Michal here: https://github.com/dotnet/runtime/issues/72645#issuecomment-1198876546. Was able to collect dumps using ulimit -c unlimited.

AaronRobinsonMSFT commented 2 years ago

Thanks @MichalStrehovsky. I've been able to reproduce this issue using your instructions on my M1 in a Docker image. I'll try to look into this more tomorrow. Has anyone tried to reproduce this using a CHK runtime?

mangod9 commented 2 years ago

no i havent tried to repro with checked. Note that this is with NativeAOT, but perhaps there is a possibility to build cg2 with checked aot?

AaronRobinsonMSFT commented 2 years ago

Oh... wait. The tool that is failing is a NativeAOT binary?

MichalStrehovsky commented 2 years ago

Oh... wait. The tool that is failing is a NativeAOT binary?

Yes, this one is for the NativeAOT compiled one. We have another one for non-AOT compiled one here but it's on Windows and happens less often: https://github.com/dotnet/runtime/issues/71702. It could be the same issue. It could be different. It wouldn't be the first issue that affects both form factors but happens more often in one.

no i havent tried to repro with checked. Note that this is with NativeAOT, but perhaps there is a possibility to build cg2 with checked aot?

Yes, if you build with build clr+libs+packs.product -c Release -rc Checked, it will build a crossgen2 binary that is compiled with a checked compiler. I haven't tried such build because the ARM64 device I have is a Raspberry Pi so I can't build directly on it, and I very likely don't have the disk space to do a crossbuild on my work machine (it took 1.5 years for Microsoft to get me a new work machine that I can't pick up right now because I'm physically located too far from it; don't ask). It is a solvable problem, but I haven't tried solving that one yet.

MichalStrehovsky commented 2 years ago

I can repro this reliable crash (reliably crashes in random spots) on ARM64 Windows as well, which is actually a good news because debugging on Windows is much better than on Linux (can do time travel debugging, etc.).

I'm submitting https://github.com/dotnet/runtime/pull/74211 to not compile crossgen2 with NativeAOT on arm64. I'll remove the blocking release label from this one. We have a bunch of other stress bugs on CoreCLR-hosted crossgen2 but it crashes a lot less often without NativeAOT and might be actually usable. I'd like to port that to RC1.

VSadov commented 2 years ago

We have a bunch of other stress bugs on CoreCLR-hosted crossgen2 but it crashes a lot less often

Do we have random crashes on CoreClr+arm64 as well?

mangod9 commented 2 years ago

We have a bunch of other stress bugs on CoreCLR-hosted crossgen2 but it crashes a lot less often

Do we have random crashes on CoreClr+arm64 as well?

Yeah we have seen regular crossgen2 crashes on arm64, but unfortunately dumps are not being captured.

a74nh commented 2 years ago

I'm seeing this failure on latest HEAD of runtime building on Arm64 Linux.

I see that it fixes when applying https://github.com/dotnet/runtime/pull/74211, but that PR was removed with https://github.com/dotnet/runtime/pull/74617

How should I be doing my Arm64 Linux builds?

(edit: fix link)

kunalspathak commented 2 years ago

I'm seeing this failure on latest HEAD of runtime building on Arm64 Linux.

What are the errors?

AntonLapounov commented 2 years ago

Yes, you can hit an intermittent failure building the repo like the one below:

  ILCompiler.RyuJit -> /runtime/artifacts/bin/ILCompiler.RyuJit/arm64/Release/ILCompiler.RyuJit.dll
  ILCompiler -> /runtime/artifacts/bin/coreclr/Linux.arm64.Release/ilc/ilc.dll
  Optimizing assemblies for size. This process might take a while.
/runtime/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(465,5): error : Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException. [/runtime/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj]
/runtime/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(353,5): error NETSDK1096: Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [/runtime/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj]

The issue is that we are using the non-fixed crossgen2 binary from the Preview 7 NuGet package instead of the live one, which is fixed.

a74nh commented 2 years ago

Yes, you can hit an intermittent failure building the repo like the one below

That's the one I'm hitting, but it's every time (HiSilicon D05 with Ubuntu 18.04)

The issue is that we are using the non-fixed crossgen2 binary from the Preview 7 NuGet package instead of the live one, which is fixed.

Is there an easy workaround to force the build to use the live version? Otherwise should we just be manually reverting https://github.com/dotnet/runtime/pull/74617 for now?

Do you know roughly when this will be fixed properly?

AntonLapounov commented 2 years ago

Otherwise should we just be manually reverting https://github.com/dotnet/runtime/pull/74617 for now?

Either that or disable using server GC (DOTNET_gcServer=0).

VSadov commented 2 years ago

disable using server GC (DOTNET_gcServer=0)

I am not sure that will work. The choice of GC mode is a compile-time thing in NativeAOT.

mangod9 commented 2 years ago

Guess we should revert https://github.com/dotnet/runtime/pull/74617 till we update the repo to use RC1.

VSadov commented 2 years ago

I do not think reverting back and forth the way crossgen2 is published has any effect on the crossgen that is actually in use, which is from Prevew 7 Nuget package.

Replacing the binaries in the nuget cache with a locally built copy might work, but is a fragile solution. I can't think of a better workaround.

(I am also annoyed by the failures, my solution is basically try again, sometimes a few times)

AntonLapounov commented 2 years ago

disable using server GC (DOTNET_gcServer=0)

I am not sure that will work. The choice of GC mode is a compile-time thing in NativeAOT.

Yes, it works: https://github.com/dotnet/runtime/blob/1d27c3da294df541478a78ae977d7629b17ae710/src/coreclr/nativeaot/Runtime/gcrhenv.cpp#L175

AntonLapounov commented 2 years ago

Guess we should revert #74617 till we update the repo to use RC1.

Note that we actually need to check whether the host architecture is ARM64 instead of checking whether the target architecture is ARM64. If you target ARM64 on an x64 machine, you don't need that workaround.

VSadov commented 2 years ago

Ah, I see, SVR build is a superset of WKS, so it is possible to force crossgen2 into WKS mode.

This would be a nice workaround!

AntonLapounov commented 2 years ago

Yep, and you can disable it in the config file just for this binary (e.g., /home/username/.nuget/packages/microsoft.netcore.app.crossgen2.linux-arm64/7.0.0-preview.7.22375.6/tools/crossgen2.runtimeconfig.json).

MichalStrehovsky commented 2 years ago

Otherwise should we just be manually reverting https://github.com/dotnet/runtime/pull/74617 for now?

I don't think reverting that one has a connection to the crossgen2 crash while crossgenning ILC that Anton described (https://github.com/dotnet/runtime/issues/72645#issuecomment-1232005657). That pull request doesn't affect the crossgen that is in use to crossgen ILC. It affects future crossgen in a future SDK.

I've submitted https://github.com/dotnet/runtime/pull/74856 to work around.

Yep, and you can disable it in the config file just for this binary (e.g., /home/username/.nuget/packages/microsoft.netcore.app.crossgen2.linux-arm64/7.0.0-preview.7.22375.6/tools/crossgen2.runtimeconfig.json).

NativeAOT doesn't read runtimeconfig.json so this is likely just placebo effect. We were generating unused runtimeconfig and deps json files with NativeAOT in the past but that has been fixed.

AntonLapounov commented 2 years ago

NativeAOT doesn't read runtimeconfig.json so this is likely just placebo effect.

Thank you for the corrections. I have never actually tried that, just assumed that if we produce a file, it should probably work. :) Setting DOTNET_gcServer=0 certainly works for me.