dotnet / linker

388 stars 126 forks source link

ILLink : error IL1012: IL Trimmer has encountered an unexpected error #3129

Closed hez2010 closed 1 year ago

hez2010 commented 1 year ago
  Fatal error in IL Linker
  Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'document')
     at Mono.Cecil.Cil.SequencePoint..ctor(Int32 offset, Document document)
     at Mono.Cecil.SignatureReader.ReadSequencePoints(Document document)
     at Mono.Cecil.MetadataReader.ReadSequencePoints(MethodDefinition method)
     at Mono.Cecil.Cil.PortablePdbReader.ReadSequencePoints(MethodDebugInformation method_info)
     at Mono.Cecil.Cil.PortablePdbReader.Read(MethodDefinition method)
     at Mono.Cecil.Cil.CodeReader.ReadMethodBody()
     at Mono.Cecil.Cil.CodeReader.ReadMethodBody(MethodDefinition method)
     at Mono.Cecil.MetadataReader.ReadMethodBody(MethodDefinition method)
     at Mono.Cecil.MethodDefinition.<>c.<get_Body>b__41_0(MethodDefinition method, MetadataReader reader)
     at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TRet& variable, TItem item, Func`3 read)
     at Mono.Cecil.MethodDefinition.get_Body()
     at Mono.Linker.Dataflow.CompilerGeneratedState.<GetCompilerGeneratedStateForType>g__ProcessMethod|11_0(MethodDefin
  ition method, <>c__DisplayClass11_0&)
     at Mono.Linker.Dataflow.CompilerGeneratedState.GetCompilerGeneratedStateForType(TypeDefinition type)
     at Mono.Linker.Dataflow.CompilerGeneratedState.TryGetCompilerGeneratedCalleesForUserMethod(MethodDefinition method
  , List`1& callees)
     at Mono.Linker.Steps.MarkStep.MarkReflectionLikeDependencies(MethodBody body, Boolean requiresReflectionMethodBody
  Scanner)
     at Mono.Linker.Steps.MarkStep.MarkMethodBody(MethodBody body)
     at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method, DependencyInfo& reason, MessageOrigin& origin
  )
     at Mono.Linker.Steps.MarkStep.ProcessQueue()
     at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue()
     at Mono.Linker.Steps.MarkStep.Process()
     at Mono.Linker.Steps.MarkStep.Process(LinkContext context)
     at Mono.Linker.Pipeline.ProcessStep(LinkContext context, IStep step)
     at Mono.Linker.Pipeline.Process(LinkContext context)
     at Mono.Linker.Driver.Run(ILogger customLogger)
     at Mono.Linker.Driver.Main(String[] args)

Repro steps:

  1. git clone --recursive https://github.com/AvaloniaUI/Avalonia
    cd samples/ControlCatalog.NetCore
  2. Change ControlCatalog.NetCore.csproj TFM to net7.0.

  3. dotnet publish -c Release -r win-x64 /p:PublishTrimmed=true

.NET SDK Version: 7.0.100

marek-safar commented 1 year ago

@vitek-karas considering your cecil symbol files knowledge ;-) Could you look into this one?

vitek-karas commented 1 year ago

@marek-safar thanks for throwing me under the bus ;-)

marek-safar commented 1 year ago

Cross linking to original issue https://github.com/AvaloniaUI/Avalonia/issues/9127#issuecomment-1325314669

vitek-karas commented 1 year ago

I'm pretty sure this is a corruption in the pdb file. The reason why this fails is that at one point it reads really large number (like 2^30) from the sequence points stream - and that leads to all kinds of weird things.

I tried to run it through a completely different tool Pdb2Pdb and that one also fails saying "Invalid compressed integer." which matches what I'm seeing in Cecil.

I tried building it for .net 6 with 6.0 SDK, but it still fails the same way.

This happens in Avalonia.Themes.Fluent.dll/pdb. When processing sequence points for method System.Void CompiledAvaloniaXaml.!AvaloniaResources::Populate:/Controls/FluentControls.xaml(System.IServiceProvider,Avalonia.Styling.Styles). It fails on sequence point #67.

This assembly seems to be generated from XAML (the sequence points point to a XAML file document), so probably some custom tool in Avalonia (sorry, don't really know how Avalonia works).

MichalStrehovsky commented 1 year ago

FWIW, we have code in the NativeAOT compiler to catch situations like this: https://github.com/dotnet/runtime/blob/9cafe8f23295b5c7f4718eaaebee6dddc2e7dda6/src/coreclr/tools/aot/ILCompiler.RyuJit/Compiler/DependencyAnalysis/MethodCodeNode.cs#L263-L269

(The referenced Roslyn bug has existed for years so there are libraries in the wild produced by Roslyn that run into this.)

The "Invalid compressed integer." message you saw in Pdb2Pdb is a BadImageFormatException from S.R.Metadata that the above code would likely catch in NativeAOT. It is likely the reason why NativeAOT doesn't choke on this (despite people using it with Avalonia). The runtime manifestation of bugs like this is usually "stepping and breakpoints don't work", so dropping the stepping information is no different.

Of course we would probably not want to catch an ArgumentNullException in illink - Cecil would have to do something more meaningful for it.

hez2010 commented 1 year ago

This assembly seems to be generated from XAML (the sequence points point to a XAML file document), so probably some custom tool in Avalonia (sorry, don't really know how Avalonia works).

Avalonia has a msbuild task which uses Mono.Cecil to emit IL code (compiled XAML) to the built assembly.

vitek-karas commented 1 year ago

I tried, but I can't get it to work such that I can debug that task. Can somebody with some experience in it please advice how to setup the build locally so that I can get the task which writes the IL code (compiled XAML) under the debugger? I got as far as:

dotnet msbuild /nodereuse:false Avalonia.Themes.Fluent.csproj /t:CompileAvaloniaXaml ...

But under the debugger this does basically nothing because it doesn't find any XAML resources to process. Which is weird, since the fully compiled assembly clearly contains code generated from XAML.

hez2010 commented 1 year ago

The task is located here: src/Avalonia.Build.Tasks

maxkatz6 commented 1 year ago

@vitek-karas hi! Just noticed this issue, and I think I can help here. TL;DR: this issue was fixed on the Avalonia side. But if you are interested in what exactly happened, I will leave details below.

Avalonia indeed has a XAML compiler which transforms XML files and then generates IL code in the same assembly with help of Mono.Cecil. It also injects debug information, so it's possible to debug XAML elements generation in the IDE. It works pretty well, and there almost no issues with linker too (only one I found yesterday, but it's not related to this ticket).

After XamlX compiler has compiler its job, in Avalonia we also replace some previously emitted code, using Mono.Cecil only. The idea is to replace reflection-based file-includes between XAML files with generated type calls.

This subtask was finding IL code (slightly simplified):

ldstr "avares://Assembly/XamlFile.xaml"
call XamlHelpers.ResolveXamlFile 

and replacing with:

nop
newobj !CompiledXaml.XamlFile

Accidently, it was also breaking debug information on its way. It was not manually adjusted and was causing error from this thread.

I have fixed this issue in my PR by moving this logic to the XamlX compiler code, so IL is emitted only once, without rewriting it after. And debug information is also inserted once by XamlX compiler and based on the final IL of the generated method.

maxkatz6 commented 1 year ago

I suppose your fix in the Cecil, https://github.com/jbevain/cecil/pull/867, could help with our IL-rewriting code. But we use stable nuget version, which is quite out of date. Also, our current solution by avoiding IL rewriting at all is more flexible and preferable in the end.

maxkatz6 commented 1 year ago

If you have any questions or need help with our code, please ask.

vitek-karas commented 1 year ago

Thanks a lot for the explanation - and for the offer.

Closing this issue as resolve.