search

dotnet / core

.NET news, announcements, release notes, and more!
https://dot.net
MIT License
20.96k stars 4.91k forks source link

Running "dotnet store" fails in crossgen #4031

Open kosmakoff opened 4 years ago

kosmakoff commented 4 years ago

I am trying to create the runtime store for my app for linux runtime.

I identified that this minimal manifest won't compile in my environment for target framework netcoreapp3.1 and runtime rhel.7.2-x64.

<Project Sdk="Microsoft.NET.Sdk">
  <ItemGroup>
    <PackageReference Include="AutoMapper" Version="9.0.0" />
  </ItemGroup>
</Project>

I use the WSL (Ubuntu) on Windows 10.

Command:

dotnet store --manifest ./store_broken.manifest --output ./store_broken_artifacts --framework netcoreapp3.1 --runtime rhel.7.2-x64

Output:

Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 436.43 ms for /mnt/c/Dev/dataart/bcp-dataart/BCP/store_broken.manifest.
  Restore completed in 214.5 ms for /mnt/c/Dev/dataart/bcp-dataart/BCP/store_broken.manifest.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.IO.dll generated successfully.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.dll generated successfully.
  Error: Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
  Error compiling /home/olegk/.nuget/packages/automapper/9.0.0/lib/netstandard2.0/AutoMapper.dll: Could not find or load a specific file. (0x80131621)
  Error: compilation failed for "/home/olegk/.nuget/packages/automapper/9.0.0/lib/netstandard2.0/AutoMapper.dll" (0x80131621)
/usr/share/dotnet/sdk/3.1.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(148,5): error MSB3073: The command "/tmp/jg4qy4hg.gcl/Optimize/netcoreapp/crossgen -nologo -readytorun -in "/home/olegk/.nuget/packages/automapper/9.0.0/lib/netstandard2.0/AutoMapper.dll" -out "/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/AutoMapper.dll" -jitpath "/home/olegk/.nuget/packages/microsoft.netcore.app.runtime.linux-x64/3.1.0/runtimes/linux-x64/native/libclrjit.so" -platform_assemblies_paths "/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimeref:/tmp/jg4qy4hg.gcl/Optimize/netcoreapp"" exited with code 33. [/mnt/c/Dev/dataart/bcp-dataart/BCP/store_broken.manifest]      Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.dll'.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.IO.dll'.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Text.Encoding.dll generated successfully.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Runtime.dll generated successfully.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Text.Encoding.dll'.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Primitives.dll generated successfully.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Threading.Tasks.dll generated successfully.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Runtime.dll'.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Primitives.dll'.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Private.Uri.dll generated successfully.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Threading.Tasks.dll'.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Private.Uri.dll'.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Emit.dll generated successfully.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Emit.dll'.
  Native image /tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Emit.ILGeneration.dll generated successfully.
  Successfully generated perfmap for native assembly '/tmp/jg4qy4hg.gcl/AutoMapper_9.0.0/runtimopt/System.Reflection.Emit.ILGeneration.dll'.

Error suggests that System.Runtime.dll cannot be found, however it is found at both paths passed to crossgen as platform_assemblies_paths parameter.

Inspecting these two System.Runtime.dll files I found that they have different versions (and sizes):

foriequal0 commented 4 years ago

I've encountered similar errors. In my case, dotnet store installs System.Runtime 4.3.0 from NuGet.org in Restore target, and _RestoreCrossgen copies 4.3.0 binary to the temporary directory to optimize. However, a DLL that built with 3.1.1, it requires 4.2.2, which is not compatible with 4.3.0, it fails to build.

It is related to the crossgen optimizer, so --skip-optimization can be a workaround of it. (However, it triggers another bug C:\Program Files\dotnet\sdk\3.1.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ComposeStore.targets(345,5): error MSB3094 which involves with a package with multiple dlls)

I think there should be dotnet 3.1.1 runtime path in the $(CrossgenPlatformAssembliesPath) which is passed to the crossgen or dotnet 3.1.1 runtime libraries should be copied to the $(CrossgenPlatformAssembliesPath)

kosmakoff commented 4 years ago

Skipping optimization helped me, but it completely defeats the point - I wanted these DLLs optimized for target environment.

I understand that crossgen binaries is not considered public API, therefore it should either work perfectly, or at least report errors that we would know how to fix on our end.

scalablecory commented 4 years ago

@jashook @safern is this something you can help with? Thanks.

safern commented 4 years ago

@davidwrighton is this something your team can help with this?

davidwrighton commented 4 years ago

I'm not familiar with dotnet store, but @mangod9 can probably assign someone to research this.

mangod9 commented 4 years ago

Yeah I can have someone investigate. @kosmakoff are there specific repro steps for this?

kosmakoff commented 4 years ago

@mangod9 I've just tried those steps from the initial post, and they are still valid, i.e. dotnet store fails on them.

  1. Have Ubuntu 18.04.3 LTS installed in WSL2 in Winows 10. On top of that there is a dotnet SDK version 3.1.100.

  2. Create the file named store_broken.manifest with following contents.

<Project Sdk="Microsoft.NET.Sdk">
  <ItemGroup>
    <PackageReference Include="AutoMapper" Version="9.0.0" />
  </ItemGroup>
</Project>
  1. Execute the command:

dotnet store --manifest ./store_broken.manifest --output ./store_broken_artifacts --framework netcoreapp3.1 --runtime rhel.7.2-x64.

  1. Observe errors:

image

After upgrading the dotnet tool via official instructions from this page dotnet reports version 3.1.101 but the issue remains.

Interestingly enough, if I choose to upgrade the Automapper to it's latest (to 10.0.0) then dotnet restore works flawlessly. So it must be something with the version 9.0.0 of the package that the dotnet restore fails to process correctly.

koliyo commented 4 years ago

Same problem here with the official mcr.microsoft.com/dotnet/core/sdk:3.1 docker image. I am using --runtime linux-x64