search

dotnet / corert

This repo contains CoreRT, an experimental .NET Core runtime optimized for AOT (ahead of time compilation) scenarios, with the accompanying compiler toolchain.
http://dot.net
MIT License
2.91k stars 508 forks source link

run-time exception with McMaster.Extensions.CommandLineUtils #6245

Open JoshEbersol opened 6 years ago

JoshEbersol commented 6 years ago

I'm able to compile a program that uses McMaster.Extensions.CommandLineUtils (version 2.2.5) like so:

public static Task<int> Main(string[] args) => CommandLineApplication.ExecuteAsync<MyProgram>(args);

private async Task<int> OnExecuteAsync()
{
    return 0;
}

...but when I run the program, I crash like this:

>MyProgram.exe --help
Unhandled Exception: System.TypeInitializationException: A type initializer threw an exception. To determine which type, inspect the InnerException's StackTrace property. ---> System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1, Func`2) + 0x1f1
   at MyProgram!<BaseAddress>+0x4ee9a
   at MyProgram!<BaseAddress>+0x133670

   --- End of inner exception stack trace ---
   at MyProgram!<BaseAddress>+0x133724
   at MyProgram!<BaseAddress>+0x133550
   at McMaster.Extensions.CommandLineUtils.Conventions.ConstructorInjectionConvention.Apply(ConventionContext) + 0x48
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Builder.McMaster.Extensions.CommandLineUtils.Conventions.IConventionBuilder.AddConvention(IConvention) + 0x3b
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseDefaultConventions(IConventionBuilder) + 0xfa
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute[TApp](CommandLineContext) + 0xb6
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](CommandLineContext) + 0xc
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](IConsole, String[]) + 0xa4
   at MyProgram.MyProgram.<Main>(String[]) + 0x13
   at MyProgram!<BaseAddress>+0x1a34aa

Same code works fine if not natively compiled.

jkotas commented 6 years ago

McMaster.Extensions.CommandLineUtils has its own dependency injection framework. The dependency injection frameworks do not work great with CoreRT AOT compilation currently. They require rd.xml file to describe types that will be needed by the dependency injection framework at runtime. https://github.com/dotnet/corert/tree/master/samples/WebApi has an example how to add rd.xml file.

In this case, you can start by adding <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All" /> to rd.xml. It will make everything in McMaster.Extensions.CommandLineUtils available to reflection. It is probably not necessary to add everything (doing so bloats the native binary), but it has may make things just work.

JoshEbersol commented 6 years ago

Thanks, @jkotas - that gets me a little bit further. I now fail like this:

MyProgram.exe --help
Unhandled Exception: System.TypeInitializationException: A type initializer threw an exception. To determine which type, inspect the InnerException's StackTrace property. ---> System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1, Func`2) + 0x1f9
   at MyProgram!<BaseAddress>+0x2c0db0
   at MyProgram!<BaseAddress>+0x8d48c6
   at MyProgram!<BaseAddress>+0x8d4331

   --- End of inner exception stack trace ---
   at MyProgram!<BaseAddress>+0x8d441d
   at MyProgram!<BaseAddress>+0x8d40b6
   at McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider.GetParser(Type) + 0x2b
   at McMaster.Extensions.CommandLineUtils.Conventions.OptionAttributeConventionBase`1.AddOption(ConventionContext, CommandOption, PropertyInfo) + 0x7a2
   at McMaster.Extensions.CommandLineUtils.Conventions.OptionAttributeConvention.Apply(ConventionContext) + 0x138
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Builder.McMaster.Extensions.CommandLineUtils.Conventions.IConventionBuilder.AddConvention(IConvention) + 0x4b
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseOptionAttributes(IConventionBuilder) + 0x40
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseAttributes(IConventionBuilder) + 0xb8
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseDefaultConventions(IConventionBuilder) + 0x5a
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute[TApp](CommandLineContext) + 0x177
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](CommandLineContext) + 0x31
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](IConsole, String[]) + 0x9d
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](String[]) + 0x42
   at MyProgram.MyProgram.Main(String[]) + 0x23
   at MyProgram.MyProgram.<Main>(String[]) + 0x2b
   at MyProgram!<BaseAddress>+0xbaec0f
   at MyProgram!<BaseAddress>+0xbaec9f

Looking at the code that fails, there's a bunch of reflection going on in there - I've tried adding quite a few additional assemblies to rd.xml (System.ComponentModel, System.Collections, System.Globalization, System.Reflection, etc.), but I haven't managed to get past this one.

mig42 commented 5 years ago

Hi,

I'm experiencing a similar issue. I'm using MacMaster.Extensions.CommandLineUtils v2.3.1 and Microsoft.DotNet.ILCompiler v1.0.0-alpha-27418-01.

My Program class looks like this:

static int Main(string[] args)
{
    return CommandLineApplication.Execute<Program>(args);
}

int OnExecute(CommandLineApplication app)
{
    // ...
}

and I'm using this rd.xml file:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All" />
  </Application>
</Directives>

I'm not getting the TypeInitializationException, probably because I'm not using an async Main(). However, I'd say that our root reason is the same because this is the exception I get:

Unhandled Exception: EETypeRva:0x00D22560(System.Reflection.MissingRuntimeArtifactException): MakeGenericMethod() cannot create this generic method instantiation because the instantiation was not metadata-enabled: 'McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider.GetParser<System.String>()' For more information, please visit http://go.microsoft.com/fwlink/?LinkID=616868
   at Internal.Reflection.Core.Execution.ExecutionEnvironment.GetMethodInvoker(RuntimeTypeInfo, QMethodDefinition, RuntimeTypeInfo[], MemberInfo) + 0x148
   at System.Reflection.Runtime.MethodInfos.NativeFormat.NativeFormatMethodCommon.GetUncachedMethodInvoker(RuntimeTypeInfo[], MemberInfo) + 0x50
   at System.Reflection.Runtime.MethodInfos.RuntimeMethodInfo.get_MethodInvoker() + 0xa8
   at System.Reflection.Runtime.MethodInfos.RuntimeNamedMethodInfo`1.MakeGenericMethod(Type[]) + 0x104
   at McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider.GetParser(Type) + 0x47
   at McMaster.Extensions.CommandLineUtils.Conventions.OptionAttributeConventionBase`1.AddOption(ConventionContext, CommandOption, PropertyInfo) + 0x37d
   at McMaster.Extensions.CommandLineUtils.Conventions.OptionAttributeConvention.Apply(ConventionContext) + 0xb0
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Builder.McMaster.Extensions.CommandLineUtils.Conventions.IConventionBuilder.AddConvention(IConvention) + 0x3c
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseAttributes(IConventionBuilder) + 0xe6
   at McMaster.Extensions.CommandLineUtils.ConventionBuilderExtensions.UseDefaultConventions(IConventionBuilder) + 0x13
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute[TApp](CommandLineContext) + 0xc2
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute[TApp](IConsole, String[]) + 0xb8
   at project!<BaseAddress>+0x6741af

You see, the method McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider.GetParser(Type) is present in both our stack traces.

Looking at the exception message, I tried to explicitly include McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider.GetParser<System.String>() in my rd.xml. However, I have no idea how to achieve that. I tried this:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All">
      <Type Name="McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider" Dynamic="Required All">
      </Type>
    </Assembly>
    <Assembly Name="mscorlib" />
  </Application>
</Directives>

and this:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All">
      <Type Name="McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider" Dynamic="Required All">
        <Method Name="GetParser">
        </Method>
      </Type>
    </Assembly>
    <Assembly Name="mscorlib" />
  </Application>
</Directives>

But none of them made any difference. Then, I tried this:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All">
      <Type Name="McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider" Dynamic="Required All">
        <Method Name="GetParser">
          <GenericArgument Name="System.String,mscorlib" />
        </Method>
      </Type>
    </Assembly>
    <Assembly Name="mscorlib" />
  </Application>
</Directives>

I had to look at https://github.com/dotnet/corert/blob/master/src/ILCompiler/src/RdXmlRootProvider.cs to avoid the System.NotSupportedExceptions I got. It seems that the troubleshooters such as https://dotnet.github.io/native/troubleshooter/method.html aren't useful, and I had to use GenericArgument inside Method despite not being listed as available child tag in the schema.

The rd.xml above produced this on compile time:

EXEC : error : Exception of type 'System.Exception' was thrown. [C:\path\to\my\project.csproj]
  System.Exception: Exception of type 'System.Exception' was thrown.
     at ILCompiler.RdXmlRootProvider.ProcessMethodDirective(IRootingServiceProvider rootProvider, ModuleDesc containingModule, TypeDesc containingType, XElement methodElement)
     at ILCompiler.RdXmlRootProvider.ProcessTypeDirective(IRootingServiceProvider rootProvider, ModuleDesc containingModule, XElement typeElement)
     at ILCompiler.RdXmlRootProvider.ProcessAssemblyDirective(IRootingServiceProvider rootProvider, XElement assemblyElement)
     at ILCompiler.RdXmlRootProvider.AddCompilationRoots(IRootingServiceProvider rootProvider)
     at ILCompiler.Compilation..ctor(DependencyAnalyzerBase`1 dependencyGraph, NodeFactory nodeFactory, IEnumerable`1 compilationRoots, ILProvider ilProvider, DebugInformationProvider debugInformationProvider, DevirtualizationManager devirtualizationManager, Logger logger)
     at ILCompiler.ILScannerBuilder.ToILScanner()
     at ILCompiler.Program.Run(String[] args)
     at ILCompiler.Program.Main(String[] args)

I thought that maybe I forgot to specify the generic type parameter in the method signature, so I tried this instead:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="McMaster.Extensions.CommandLineUtils" Dynamic="Required All">
      <Type Name="McMaster.Extensions.CommandLineUtils.Abstractions.ValueParserProvider" Dynamic="Required All">
        <Method Name="GetParser`1">
          <GenericArgument Name="System.String,mscorlib" />
        </Method>
      </Type>
    </Assembly>
    <Assembly Name="mscorlib" />
  </Application>
</Directives>

And then I got a NullReferenceException:

EXEC : error : Object reference not set to an instance of an object. [C:\path\to\my\project.csproj]
  System.NullReferenceException: Object reference not set to an instance of an object.
     at ILCompiler.RdXmlRootProvider.ProcessMethodDirective(IRootingServiceProvider rootProvider, ModuleDesc containingModule, TypeDesc containingType, XElement methodElement)
     at ILCompiler.RdXmlRootProvider.ProcessTypeDirective(IRootingServiceProvider rootProvider, ModuleDesc containingModule, XElement typeElement)
     at ILCompiler.RdXmlRootProvider.ProcessAssemblyDirective(IRootingServiceProvider rootProvider, XElement assemblyElement)
     at ILCompiler.RdXmlRootProvider.AddCompilationRoots(IRootingServiceProvider rootProvider)
     at ILCompiler.Compilation..ctor(DependencyAnalyzerBase`1 dependencyGraph, NodeFactory nodeFactory, IEnumerable`1 compilationRoots, ILProvider ilProvider, DebugInformationProvider debugInformationProvider, DevirtualizationManager devirtualizationManager, Logger logger)
     at ILCompiler.ILScannerBuilder.ToILScanner()
     at ILCompiler.Program.Run(String[] args)
     at ILCompiler.Program.Main(String[] args)

I got the same if I used GetParser`1[[System.Object,mscorlib]] as the method name.

Is there anything else that I could try out? What is the proper way to enable metadata for that generic method?

jkotas commented 5 years ago

Is there anything else that I could try out?

Try using GetParser as the method name. The generic arity suffix is used for types only.

Alternatively, you can consider using command line parser that is more AOT friendly, e.g. https://github.com/dotnet/command-line-api.

mig42 commented 5 years ago

Unfortunately I already tried that... I got a System.Exception() with no message, you can see the trace in my previous comment. Or did I miss something in that one?

I'll look into the link you proposed, thanks!

MichalStrehovsky commented 5 years ago

I'm pretty sure it's matching the non-generic version of the GetParser method. Not being able to specify overloads is a known limitation.

No RD.XML should be needed once we merge #6987. It should work if you build the repo yourself with that change. We can't merge it yet because it causes CI to fail (it makes us discover generic CoreFX tests we weren't running before). With any luck, I'll get to it this weekend.

mig42 commented 5 years ago

Thanks for the answer! I'll stay tuned to the new releases then, I appreciate your feedback 😃