search

dotnet / runtimelab

This repo is for experimentation and exploring new ideas that may or may not make it into the main dotnet/runtime repo.
MIT License
1.43k stars 200 forks source link

[Bug] [NativeAOT-LLVM]: Unable to compile native WebAssembly library #2073

Closed just-ero closed 1 year ago

just-ero commented 2 years ago

Description

Attempted compilation of a native WASM library aborts with System.AccessViolationException:

Determining projects to restore...
  All projects are up-to-date for restore.
  NativeWasm -> ~\bin\Debug\net7.0\browser-wasm\NativeWasm.dll
  Generating compatible native code.

/_/src/libraries/System.Private.CoreLib/src/System/Resources/ManifestBasedResourceGroveler.cs(238): Trim analysis warning IL2026: System.Resources.ManifestBasedResourceGroveler.CreateResourceSet(Stream,Assembly): Using method 'System.Resources.ManifestBasedResourceGroveler.InternalGetResourceSetFromSerializedData(Stream,String,String,ResourceManagerMediator)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. The CustomResourceTypesSupport feature switch has been enabled for this app which is being trimmed. Custom readers as well as custom objects on the resources file are not observable by the trimmer and so required assemblies, types and members may be removed.

  Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
  Repeat 2 times:
  --------------------------------
     at LLVMSharp.Interop.LLVM.BuildLoad(LLVMSharp.Interop.LLVMOpaqueBuilder*, LLVMSharp.Interop.LLVMOpaqueValue*, SByte*)
  --------------------------------
     at LLVMSharp.Interop.LLVMBuilderRef.BuildLoad(LLVMSharp.Interop.LLVMValueRef, System.ReadOnlySpan`1<Char>)
     at Internal.IL.ILImporter.LoadAddressOfSymbolNode(ILCompiler.DependencyAnalysis.ISymbolNode, LLVMSharp.Interop.LLVMBuilderRef)
     at Internal.IL.ILImporter.GetFieldAddress(Internal.TypeSystem.FieldDesc, Internal.TypeSystem.FieldDesc, Boolean)
     at Internal.IL.ILImporter.ImportLoadField(Int32, Boolean)
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at Internal.IL.ILImporter.CompileMethod(ILCompiler.LLVMCodegenCompilation, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
     at ILCompiler.LLVMCodegenCompilation.CompileSingleMethod(Internal.JitInterface.CorInfoImpl, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
     at ILCompiler.LLVMCodegenCompilation.CompileSingleThreaded(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysis.LLVMMethodCodeNode>)
     at ILCompiler.LLVMCodegenCompilation.ComputeDependencyNodeDependencies(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<ILCompiler.DependencyAnalysis.NodeFacto
  ry>>)
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2[[ILCompiler.DependencyAnalysisFramework.NoLogStrategy`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyT
  oken=7cec85d7bea7798e]], ILCompiler.DependencyAnalysisFramework, Version=7.0.0.0, Culture=neutral, PublicKeyToken=null],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyTok
  en=7cec85d7bea7798e]].ComputeMarkedNodes()
     at ILCompiler.LLVMCodegenCompilation.CompileInternal(System.String, ILCompiler.ObjectDumper)
     at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(System.String, ILCompiler.ObjectDumper)
     at ILCompiler.Program.Run(System.String[])
     at ILCompiler.Program.Main(System.String[])
~\.nuget\packages\microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22528.1\build\Microsoft.NETCore.Native.targets(309,5): error MSB3073:
The command ""~\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22528.1\tools\ilc" @"obj\Debug\net7.0\browser-wasm\native\AutoSplitRuntimeInterop.ilc.rsp"" exited with code -1073741819.

(Paths shortened for conciseness).

Minimal repro

All that's required is a project file (package feed is https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json).

.csproj 
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net7.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM; runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM" Version="7.0.0-alpha.1.22528.1" />
  </ItemGroup>

</Project>
Powershell 
> dotnet publish /p:NativeLib=Static /p:SelfContained=true -r browser-wasm -c Debug /p:TargetArchitecture=wasm /p:PlatformTarget=AnyCPU /p:MSBuildEnableWorkloadResolver=false /p:EmccExtraArgs="-s EXPORTED_RUNTIME_METHODS=cwrap" --self-contained
yowl commented 2 years ago

@jkotas I'm not sure the packages are publishing, if you get a moment, can you see why? https://dev.azure.com/dnceng/public/_artifacts/feed/dotnet-experimental shows a date in October

image

I did try to reproduce this with the head of the branch, and it compiled successfully. So hopefully it was fixed by some commit recently and is just a question of getting the latest packages out.

jkotas commented 2 years ago

Looking into this. It is stuck on some sort of permission problem.

jkotas commented 2 years ago

It is more VS2019 references that needs cleaning up. Prepare for Publish fails with:

##[error]Failed to request agent. Exception Image build.windows.10.amd64.vs2019 doesn't exist in pool NetCore1ESPool-Internal
yowl commented 2 years ago

Thanks, I probably should have grepped for that.

yowl commented 2 years ago

Either its not finished or there's more to update image

jkotas commented 2 years ago

There are more permission issues that I am working through.

jkotas commented 2 years ago

A new build has been published. Give it a try!

just-ero commented 2 years ago

A new build has been published. Give it a try!

The same issue still persists.

.csproj 
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net7.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="7.0.0-alpha.1.22564.2" />
    <PackageReference Include="runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM" Version="7.0.0-alpha.1.22564.2" />
  </ItemGroup>

</Project>
Powershell 
> dotnet publish /p:NativeLib=Static /p:SelfContained=true -r browser-wasm -c Debug /p:TargetArchitecture=wasm /p:PlatformTarget=AnyCPU /p:MSBuildEnableWorkloadResolver=false --sc

Output:

MSBuild version 17.4.0+18d5aef85 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
  NativeWasm -> C:\Users\Ero\Desktop\NativeWasm\bin\Debug\net7.0\browser-wasm\NativeWasm.dll
  Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
  Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeCoreRT
  Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
  Repeat 2 times:
  --------------------------------
     at LLVMSharp.Interop.LLVM.BuildLoad(LLVMSharp.Interop.LLVMOpaqueBuilder*, LLVMSharp.Interop.LLVMOpaqueValue*, SByte*)
  --------------------------------
     at LLVMSharp.Interop.LLVMBuilderRef.BuildLoad(LLVMSharp.Interop.LLVMValueRef, System.ReadOnlySpan`1<Char>)
     at Internal.IL.ILImporter.LoadAddressOfSymbolNode(ILCompiler.DependencyAnalysis.ISymbolNode, LLVMSharp.Interop.LLVMBuilderRef)
     at Internal.IL.ILImporter.GetFieldAddress(Internal.TypeSystem.FieldDesc, Internal.TypeSystem.FieldDesc, Boolean)
     at Internal.IL.ILImporter.ImportLoadField(Int32, Boolean)
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock)
     at Internal.IL.ILImporter.ImportBasicBlocks()
     at Internal.IL.ILImporter.Import()
     at Internal.IL.ILImporter.CompileMethod(ILCompiler.LLVMCodegenCompilation, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
     at ILCompiler.LLVMCodegenCompilation.CompileSingleMethod(Internal.JitInterface.CorInfoImpl, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
     at ILCompiler.LLVMCodegenCompilation.CompileSingleThreaded(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysis.LLVMMethodCodeNode>)
     at ILCompiler.LLVMCodegenCompilation.ComputeDependencyNodeDependencies(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysisFramework.DependencyNo
  deCore`1<ILCompiler.DependencyAnalysis.NodeFactory>>)
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2[[ILCompiler.DependencyAnalysisFramework.NoLogStrategy`1[[System.__Canon, System.Private.CoreL
  ib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ILCompiler.DependencyAnalysisFramework, Version=7.0.0.0, Culture=neutral, PublicKeyTok
  en=null],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ComputeMarkedNodes()
     at ILCompiler.LLVMCodegenCompilation.CompileInternal(System.String, ILCompiler.ObjectDumper)
     at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(System.String, ILCompiler.ObjectDumper)
     at ILCompiler.Program.Run(System.String[])
     at ILCompiler.Program.Main(System.String[])
C:\Users\Ero\.nuget\packages\microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\build\Microsoft.NETCore.Native.targets(313,5): error MSB3073: The command ""C
:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\\ilc" @"obj\Debug\net7.0\browser-wasm\native\NativeWasm.
ilc.rsp"" exited with code -1073741819. [C:\Users\Ero\Desktop\NativeWasm\NativeWasm.csproj]
yowl commented 2 years ago

I'm not getting that error here, so not sure what the difference is.

However I think you will hit another problem with the integration not finding the runtime location and you'll get an error like

emcc : error : /sdk/libPortableRuntime.a: No such file or directory ("/sdk/libPortableRuntime.a" was expected to be an input file, based on the commandline arguments provided) [E:\tmp\NativeAOTLLVMTest\t\t.csproj]
E:\tmp\NativeAOTLLVMTest\t\.packages\microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\build\Microsoft.NETCore.Native.targets(437,5): error MSB3073: The command ""E:/GitHub/emsdk/upstream/emscripten/emcc.bat" "obj\De
bug\net7.0\browser-wasm\native\t.o" "obj\Debug\net7.0\browser-wasm\native\tclrjit.o" -o "bin\Debug\net7.0\browser-wasm\native\t.html" -s ALLOW_MEMORY_GROWTH=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s DISABLE_EXCEPTION_CATCHING=
0 "/sdk/libPortableRuntime.a" "/sdk/libbootstrapperdll.a" "/framework/libSystem.Native.a" "/framework/libSystem.Globalization.Native.a"  -g3" exited with code 1. [E:\tmp\NativeAOTLLVMTest\t\t.csproj]
just-ero commented 2 years ago

Any way I can assist in figuring it out?

yowl commented 2 years ago

Can you post your NativeWasm.dll and rsp file?

just-ero commented 2 years ago

Can you post your NativeWasm.dll and rsp file?

I don't have an rsp file. What is that?

Download of the .dll here; NativeWasm.zip (GitHub does not support uploads of lone .dll files).

yowl commented 2 years ago

Your rsp file is here: obj\Debug\net7.0\browser-wasm\native\NativeWasm.ilc.rsp

Its the arguments passed to Ilc, the IL -> LLVM compiler, which is failing for you. I tried your dll and it compiled successfully unfortunately. If you can post the rsp file then I can try that, it's unlikely to be different to mine, but running out of ideas.

just-ero commented 2 years ago

Since the browser-wasm build errors, no native folder is even generated.

yowl commented 2 years ago

Strange as it is on your output:

:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\\ilc" @"obj\Debug\net7.0\browser-wasm\native\NativeWasm.
ilc.rsp"" exited with code -1073741819. [C:\Users\Ero\Desktop\NativeWasm\NativeWasm.csproj]

maybe it is cleaned up...

just-ero commented 2 years ago

Sorry, I overlooked the fact that it's meant to be in the obj folder, not bin. Here it is.

yowl commented 2 years ago

I tried E:\tmp\NativeAOTLLVMTest\t\.packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\ilc.exe @e:\tmp\NativeAOTLLVMTest\t\NativeWasm.ilc.rsp and it works here. Can you try to replace with your paths?

yowl commented 2 years ago

This is a WIndows x64 box I assume?

just-ero commented 2 years ago

I tried E:\tmp\NativeAOTLLVMTest\t\.packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\ilc.exe @e:\tmp\NativeAOTLLVMTest\t\NativeWasm.ilc.rsp and it works here. Can you try to replace with your paths?

Which paths am I replacing, and with what? What am I trying?

This is a WIndows x64 box I assume?

Yes it is, Win11 Pro, 10.0.22621.

yowl commented 2 years ago

E:\tmp\NativeAOTLLVMTest\t\.packages with the location of your packages E:\tmp\NativeAOTLLVMTest\t with the locaiton of your rsp

just-ero commented 2 years ago

Forgive my ignorance, but where am I putting those paths?

yowl commented 2 years ago

Looking at your rsp file from a command window, cd to your project folder and enter:

C:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\\ilc @"obj\Debug\net7.0\browser-wasm\native\NativeWasm.ilc.rsp"

I want to confirm that on its own will fail, as here that succeeds. if it fails, please zip up C:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools

Thanks

just-ero commented 2 years ago 
C:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\\ilc @"obj\Debug\net7.0\browser-wasm\native\NativeWasm.ilc.rsp"

This resulted in:

ParserError:
Line |
   1 |  … t.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\\ilc @"obj\Debug …
     |                                                                ~
     | No characters are allowed after a here-string header but before the end of the line.
C:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\ilc .\obj\Debug\net7.0\browser-wasm\native\NativeWasm.ilc.rsp

This resulted in:

Error: Output filename must be specified (/out <file>)
Internal.CommandLine.CommandLineException: Output filename must be specified (/out <file>)
   at ILCompiler.Program.Run(String[] args)
   at ILCompiler.Program.Main(String[] args)

7.0.0-alpha.1.22564.2-tools.zip (Google Drive link, 56 MB were too lage for GitHub).

yowl commented 2 years ago

You need to put an @ in front of the path to the rsp

just-ero commented 2 years ago

Did I not, in the first example?

yowl commented 2 years ago

Yes, but there seemed to be a problem with the quotes, so try it with the second example please.

just-ero commented 2 years ago 
C:\Users\Ero\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler.llvm\7.0.0-alpha.1.22564.2\tools\ilc "@.\obj\Debug\net7.0\browser-wasm\native\NativeWasm.ilc.rsp"
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:
--------------------------------
   at LLVMSharp.Interop.LLVM.BuildLoad(LLVMSharp.Interop.LLVMOpaqueBuilder*, LLVMSharp.Interop.LLVMOpaqueValue*, SByte*)
--------------------------------
   at LLVMSharp.Interop.LLVMBuilderRef.BuildLoad(LLVMSharp.Interop.LLVMValueRef, System.ReadOnlySpan`1<Char>)
   at Internal.IL.ILImporter.LoadAddressOfSymbolNode(ILCompiler.DependencyAnalysis.ISymbolNode, LLVMSharp.Interop.LLVMBuilderRef)
   at Internal.IL.ILImporter.GetFieldAddress(Internal.TypeSystem.FieldDesc, Internal.TypeSystem.FieldDesc, Boolean)
   at Internal.IL.ILImporter.ImportLoadField(Int32, Boolean)
   at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock)
   at Internal.IL.ILImporter.ImportBasicBlocks()
   at Internal.IL.ILImporter.Import()
   at Internal.IL.ILImporter.CompileMethod(ILCompiler.LLVMCodegenCompilation, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
   at ILCompiler.LLVMCodegenCompilation.CompileSingleMethod(Internal.JitInterface.CorInfoImpl, ILCompiler.DependencyAnalysis.LLVMMethodCodeNode)
   at ILCompiler.LLVMCodegenCompilation.CompileSingleThreaded(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysis.LLVMMethodCodeNode>)
   at ILCompiler.LLVMCodegenCompilation.ComputeDependencyNodeDependencies(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<ILCompiler.DependencyAnalysis.NodeFactory>>)
   at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2[[ILCompiler.DependencyAnalysisFramework.NoLogStrategy`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ILCompiler.DependencyAnalysisFramework, Version=7.0.0.0, Culture=neutral, PublicKeyToken=null],[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ComputeMarkedNodes()
   at ILCompiler.LLVMCodegenCompilation.CompileInternal(System.String, ILCompiler.ObjectDumper)
   at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(System.String, ILCompiler.ObjectDumper)
   at ILCompiler.Program.Run(System.String[])
   at ILCompiler.Program.Main(System.String[])
ChristianWeyer commented 1 year ago

Did you solve this @just-ero ?

just-ero commented 1 year ago

Did you solve this @just-ero ?

No I did not, unfortunately.

yowl commented 1 year ago

Small update, the package publishing seems to have stopped

image

Checking....

yowl commented 1 year ago

Please try with the latest packages. There seems to be an issue with time, but if you stay away from that it should be ok

E:\tmp\nativelib>dotnet publish /p:NativeLib=Static /p:SelfContained=true -r browser-wasm -c Debug /p:TargetArchitecture=wasm /p:PlatformTarget=AnyCPU /p:MSBuildEnableWorkloadResolver=false /p:EmccExtraArgs="-s EXPORTED_RUNTIME_METHODS=cwrap" --self-contained
MSBuild version 17.4.0+18d5aef85 for .NET
  Determining projects to restore...
  Restored E:\tmp\nativelib\1.csproj (in 2.7 sec).
  1 -> E:\tmp\nativelib\bin\Debug\net7.0\browser-wasm\1.dll
  Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeCoreRT
  RyuJIT compilation results, total methods 15284 RyuJit Methods 10333 67.6066%
wasm-ld : warning : function signature mismatch: time [E:\tmp\nativelib\1.csproj]
  >>> defined as (i32) -> i32 in E:\tmp\nativelib\.packages\runtime.browser-wasm.microsoft.dotnet.ilcompiler.llvm\7.0.0
  -alpha.1.22617.2\framework\libSystem.Native.a(pal_random.c.o)
  >>> defined as (i32) -> i64 in E:\GitHub\emsdk\upstream\emscripten\cache\sysroot\lib\wasm32-emscripten\libc-debug.a(e
  mscripten_time.o)
  1 -> E:\tmp\nativelib\bin\Debug\net7.0\browser-wasm\publish\

image

just-ero commented 1 year ago 
dotnet publish /p:NativeLib=Static /p:SelfContained=true -r browser-wasm -c Debug /p:TargetArchitecture=wasm /p:PlatformTarget=AnyCPU /p:MSBuildEnableWorkloadResolver=false /p:EmccExtraArgs="-s EXPORTED_RUNTIME_METHODS=cwrap" --self-contained
MSBuild version 17.4.0+18d5aef85 for .NET
  Determining projects to restore...
  Restored C:\Users\Ero\Desktop\NativeWasm\NativeWasm.csproj (in 3.01 sec).
  NativeWasm -> C:\Users\Ero\Desktop\NativeWasm\bin\Debug\net7.0\browser-wasm\NativeWasm.dll
  Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeCoreRT
  RyuJIT compilation results, total methods 15296 RyuJit Methods 10338 67.5863%
  Emscripten not found, not linking WebAssembly. To enable WebAssembly linking, install Emscripten and ensure the EMSDK environment variable points to the directory containing ups
  tream/emscripten/emcc.bat
C:\Program Files\dotnet\sdk\7.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(254,5): error MSB3030: Could not copy the file "bin\Debug\net7.0\browser-wasm\nati
ve\NativeWasm.html" because it was not found. [C:\Users\Ero\Desktop\NativeWasm\NativeWasm.csproj]

Please excuse my ignorance, but where does it expect Emscripten to be installed? I followed the download instructions and used ./emsdk activate latest --permanent.

ChristianWeyer commented 1 year ago

Make sure that emcc is on your PATH.

just-ero commented 1 year ago

emcc is set on my PATH;

> emcc
shared:INFO: (Emscripten: Running sanity checks)
emcc: error: no input files

The same error message appears.

just-ero commented 1 year ago

Alright, it appears that ./emsdk activate latest --permanent is broken.

I'm happy to report that after resolving that issue, my project now publishes successfully.