search

dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
35.39k stars 10k forks source link

Blazor RunAOTCompilation fails during BlazorWebAssembly.ComputeBlazorPublishAssets.Execute #35633

Closed pbiggar closed 3 years ago

pbiggar commented 3 years ago

Describe the bug

I have a blazor project that successfully compiles using dotnet build under .NET6. However, I cannot get it to build with RunAOTCompilation (even when IsTrimmable is set to false). It fails each time when running

dotnet publish -c Release

The failure is

Excerpt:

/usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : System.ArgumentException: An item with the same key has already been added. Key: /home/dark/.nuget/packages/fsharp.core/5.0.3-beta.21352.5/contentFiles/any/netstandard2.0/FSharp.Core.xml [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]
/usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error :    at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]
/usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error :    at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]
/usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error :    at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.GroupResolvedFilesToPublish(Dictionary`2 resolvedFilesToPublishToRemove, Dictionary`2 resolvedAssemblyToPublish, Dictionary`2 satelliteAssemblyToPublish, Dictionary`2 resolvedSymbolsToPublish, Dictionary`2 resolvedNativeAssetToPublish) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]
/usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error :    at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.Execute() [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]

See below for the full build output.

To Reproduce

The build failures can be seen in this [CircleCI]() run

Exceptions (if any)

Full build output:

dotnet publish -c Release src/Wasm/Wasm.fsproj Microsoft (R) Build Engine version 17.0.0-preview-21416-02+cb3144483 for .NET Copyright (C) Microsoft Corporation. All rights reserved. Determining projects to restore... All projects are up-to-date for restore. You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview Tablecloth -> /home/dark/app/fsharp-backend/Build/out/Tablecloth/Release/net6.0/Tablecloth.dll Prelude -> /home/dark/app/fsharp-backend/Build/out/Prelude/Release/net6.0/Prelude.dll LibExecution -> /home/dark/app/fsharp-backend/Build/out/LibExecution/Release/net6.0/linux-x64/LibExecution.dll LibExecutionStdLib -> /home/dark/app/fsharp-backend/Build/out/LibExecutionStdLib/Release/net6.0/linux-x64/LibExecutionStdLib.dll Wasm -> /home/dark/app/fsharp-backend/Build/out/Wasm/Release/net6.0/Wasm.dll Wasm (Blazor output) -> /home/dark/app/fsharp-backend/Build/out/Wasm/Release/net6.0/wwwroot Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink AOT'ing 66 assemblies Compiling native assets with emcc. This may take a while ... Compiling assembly bitcode files... [33/66] skipped unchanged files [34/66] System.Drawing.dll.bc -> System.Drawing.dll.o [took 0.142s] [35/66] System.Net.Requests.dll.bc -> System.Net.Requests.dll.o [took 0.147s] [36/66] System.Memory.dll.bc -> System.Memory.dll.o [took 0.152s] [37/66] System.IO.dll.bc -> System.IO.dll.o [took 0.153s] [38/66] System.Globalization.dll.bc -> System.Globalization.dll.o [took 0.152s] [39/66] System.IO.Compression.dll.bc -> System.IO.Compression.dll.o [took 3.348s] [40/66] System.IO.FileSystem.dll.bc -> System.IO.FileSystem.dll.o [took 0.127s] [41/66] System.Linq.dll.bc -> System.Linq.dll.o [took 4.380s] [42/66] System.Net.Primitives.dll.bc -> System.Net.Primitives.dll.o [took 0.187s] [43/66] System.Reflection.Emit.Lightweight.dll.bc -> System.Reflection.Emit.Lightweight.dll.o [took 0.184s] [44/66] System.Net.Http.dll.bc -> System.Net.Http.dll.o [took 11.659s] [45/66] System.Net.HttpListener.dll.bc -> System.Net.HttpListener.dll.o [took 0.247s] [46/66] FSharp.Control.AsyncSeq.dll.bc -> FSharp.Control.AsyncSeq.dll.o [took 17.177s] [47/66] FSharpx.Collections.dll.bc -> FSharpx.Collections.dll.o [took 19.146s] [48/66] Microsoft.CSharp.dll.bc -> Microsoft.CSharp.dll.o [took 28.271s] [49/66] System.Collections.Concurrent.dll.bc -> System.Collections.Concurrent.dll.o [took 1.305s] [50/66] LibExecutionStdLib.dll.bc -> LibExecutionStdLib.dll.o [took 37.737s] [51/66] Prelude.dll.bc -> Prelude.dll.o [took 7.312s] [52/66] Tablecloth.dll.bc -> Tablecloth.dll.o [took 10.942s] [53/66] FSharpx.Extras.dll.bc -> FSharpx.Extras.dll.o [took 50.025s] [54/66] Wasm.dll.bc -> Wasm.dll.o [took 11.995s] [55/66] System.Data.Common.dll.bc -> System.Data.Common.dll.o [took 39.713s] [56/66] System.Drawing.Primitives.dll.bc -> System.Drawing.Primitives.dll.o [took 4.068s] [57/66] LibExecution.dll.bc -> LibExecution.dll.o [took 56.201s] [58/66] FSharp.Core.dll.bc -> FSharp.Core.dll.o [took 92.315s] [59/66] System.Private.Xml.dll.bc -> System.Private.Xml.dll.o [took 107.086s] [60/66] System.Reflection.Emit.ILGeneration.dll.bc -> System.Reflection.Emit.ILGeneration.dll.o [took 0.135s] [61/66] System.Linq.Expressions.dll.bc -> System.Linq.Expressions.dll.o [took 111.342s] [62/66] Newtonsoft.Json.dll.bc -> Newtonsoft.Json.dll.o [took 45.013s] [63/66] System.Linq.Queryable.dll.bc -> System.Linq.Queryable.dll.o [took 2.378s] [64/66] System.Private.CoreLib.dll.bc -> System.Private.CoreLib.dll.o [took 116.294s] [65/66] FSharpPlus.dll.bc -> FSharpPlus.dll.o [took 74.010s] [66/66] FSharpx.Async.dll.bc -> FSharpx.Async.dll.o [took 12.235s] Linking with emcc. This may take a while ... "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/bin/wasm-ld" @/tmp/emscripten_gf3oypbh.rsp "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/bin/wasm-emscripten-finalize" --minimize-wasm-changes -g --dyncalls-i64 /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -o /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm --detect-features "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/src/compiler.js /tmp/tmp1g0uwa09.txt "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/bin/wasm-opt" --strip-dwarf --post-emscripten --no-exit-runtime -Oz --low-memory-unused --zero-filled-memory --strip-debug --strip-producers /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -o /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -g --mvp-features --enable-mutable-globals "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/tools/acorn-optimizer.js /tmp/emscripten_temp_t05i9iyg/dotnet.js AJSDCE minifyWhitespace "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/tools/acorn-optimizer.js /tmp/tmpq0vphttx.js emitDCEGraph noPrint "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/bin/wasm-metadce" --graph-file=/tmp/tmpduwrebet.txt /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -o /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -g --mvp-features --enable-mutable-globals "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/tools/acorn-optimizer.js /tmp/tmp9djbizrk.js applyDCEGraphRemovals minifyWhitespace "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/tools/acorn-optimizer.js /tmp/emscripten_temp_t05i9iyg/dotnet.js.jso.js.jso.js AJSDCE minifyWhitespace "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/bin/wasm-opt" --strip-dwarf --minify-imports-and-exports-and-modules /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -o /home/dark/app/fsharp-backend/Build/obj/Wasm/Release/net6.0/wasm/dotnet.wasm -g --mvp-features --enable-mutable-globals "/usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Node.linux-x64/6.0.0-rc.1.21419.1/tools/bin/node" /usr/share/dotnet/packs/Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.linux-x64/6.0.0-rc.1.21419.1/tools/emscripten/tools/acorn-optimizer.js /tmp/tmpzd_e996g.js applyImportAndExportNameChanges minifyWhitespace Optimizing dotnet.wasm ... /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : System.ArgumentException: An item with the same key has already been added. Key: /home/dark/.nuget/packages/fsharp.core/5.0.3-beta.21352.5/contentFiles/any/netstandard2.0/FSharp.Core.xml [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj] /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj] /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj] /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.GroupResolvedFilesToPublish(Dictionary`2 resolvedFilesToPublishToRemove, Dictionary`2 resolvedAssemblyToPublish, Dictionary`2 satelliteAssemblyToPublish, Dictionary`2 resolvedSymbolsToPublish, Dictionary`2 resolvedNativeAssetToPublish) [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj] /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/Sdks/Microsoft.NET.Sdk.BlazorWebAssembly/targets/Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(446,5): error : at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.Execute() [/home/dark/app/fsharp-backend/src/Wasm/Wasm.fsproj]

Further technical details

.NET SDK (reflecting any global.json): Version: 6.0.100-rc.1.21417.9 Commit: cec30acbd9

Runtime Environment: OS Name: ubuntu OS Version: 20.04 OS Platform: Linux RID: ubuntu.20.04-x64 Base Path: /usr/share/dotnet/sdk/6.0.100-rc.1.21417.9/

Host (useful for support): Version: 6.0.0-rc.1.21415.6 Commit: fde6b37e98

.NET SDKs installed: 6.0.100-rc.1.21417.9 [/usr/share/dotnet/sdk]

.NET runtimes installed: Microsoft.AspNetCore.App 6.0.0-rc.1.21416.9 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 6.0.0-rc.1.21415.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs: https://aka.ms/dotnet-download

javiercn commented 3 years ago

@pbiggar thanks for contacting us.

Can you give this a try with a nightly RC1 build and tell us if you are still running into it?

It would help us if you can create a binlog of the build process and share it with us.

pbiggar commented 3 years ago

Will do, gimme a few days.

johngsandiford commented 3 years ago

FYI, I get exactly the same error.

javiercn commented 3 years ago

@johngsandiford can you provide a minimal repro project?

pbiggar commented 3 years ago

Getting the same error on 6.0.100-rc.2.21460.33, will get you a binlog shortly.

pbiggar commented 3 years ago

binlog.zip

Binlog attached

johngsandiford commented 3 years ago

@johngsandiford can you provide a minimal repro project?

@javiercn Sure. The project I am working on is quite complex and private, so trying to repro with something simpler. No luck so far.

javiercn commented 3 years ago

@pbiggar The issue seems to be something in your build is causing a duplicate file to show up in the list of ResolvedFilesToPublish and that is triggering an error on one of the tasks.

image

@johngsandiford Can you at least provide the error and the stack trace from the binlog?

javiercn commented 3 years ago

There's an SDK PR out here and hopefully will make it into RC2

johngsandiford commented 3 years ago

@pbiggar The issue seems to be something in your build is causing a duplicate file to show up in the list of ResolvedFilesToPublish and that is triggering an error on one of the tasks.

image

@johngsandiford Can you at least provide the error and the stack trace from the binlog?

@javiercn binlog shared in private rep. Thanks, John

javiercn commented 3 years ago

@johngsandiford unfortunately we can't access private repos. If you can't share the binlog, at least it would help if you share the sanitized call stack you are receiving and the exact error message you are seeing, to help determine if its the same issue as @pbiggar or is a different one.

pbiggar commented 3 years ago

@pbiggar The issue seems to be something in your build is causing a duplicate file to show up in the list of ResolvedFilesToPublish and that is triggering an error on one of the tasks.

Thanks! Is this something you're looking into, or something I need to address in my build? If the latter, would appreciate a pointer on how to start if you have any!

javiercn commented 3 years ago

@pbiggar I think both. I don't think it is right for the list of ResolvedFilesToPublish to contain duplicates, at the same time, we can be a bit more flexible, ignore and log them to avoid issues like this from causing trouble.

@pbiggar I suspect you need to track where FSharp.Core.xml is added twice (probably as part of the F# specific targets) and make a fix there if possible.

On our side, I have a PR out that identifies the duplicate items, logs a message and ignores it.

pbiggar commented 3 years ago

@javiercn Wonderful, thank you!

johngsandiford commented 3 years ago

@johngsandiford unfortunately we can't access private repos. If you can't share the binlog, at least it would help if you share the sanitized call stack you are receiving and the exact error message you are seeing, to help determine if its the same issue as @pbiggar or is a different one.

@javiercn it is private but I have given you access to it.

Call stack below. Thanks, John

C:\Program Files\dotnet\sdk\6.0.100-preview.7.21379.14\Sdks\Microsoft.NET.Sdk.BlazorWebAssembly\targets\Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(398,5): Error : System.ArgumentException: An item with the same key has already been added. at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add) at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.GroupExistingStaticWebAssets(Dictionary2 assemblyAssets, Dictionary2 nativeAssets, Dictionary2 satelliteAssemblyAssets, Dictionary2 symbolAssets, Dictionary2 compressedRepresentations) at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.Execute()

javiercn commented 3 years ago

@johngsandiford don't you get the key that is causing the issue?

Also, could you try with a nightly build, we've fixed a lot of things since preview7.

javiercn commented 3 years ago

Fixed in https://github.com/dotnet/sdk/commits/e0d76e0c589d64aa1c4589e352e94e61feedf720

johngsandiford commented 3 years ago

This is not fixed in RC2. Please see output below. We have spent a lot of time trying to repro on a simple project, but without knowing what the key is, in a large project such as ours it has not yet been possible. We would actually much prefer that an error is raised (with the key) than allowing duplicates which could compromise our software. Thanks, John

C:\Program Files\dotnet\sdk\6.0.100-rc.2.21505.57\Sdks\Microsoft.NET.Sdk.BlazorWebAssembly\targets\Microsoft.NET.Sdk.BlazorWebAssembly.6_0.targets(404,5): Error : System.ArgumentException: An item with the same key has already been added. at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add) at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.GroupExistingStaticWebAssets(Dictionary2 assemblyAssets, Dictionary2 nativeAssets, Dictionary2 satelliteAssemblyAssets, Dictionary2 symbolAssets, Dictionary2 compressedRepresentations) at Microsoft.NET.Sdk.BlazorWebAssembly.ComputeBlazorPublishAssets.Execute()

pbiggar commented 2 years ago

Confirmed that this works for me in the released .NET6. Thanks!