[9.0.100-rc.2.24423.15] StaticWebAssets SDK - Conflicting assets in the same target path

[robertmclaws]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the bug

The new StaticWebAssets SDK continues to be causing numerous issues for customers.

This time seems similar to this issue around compression. Referencing a NuGet-based RCL in a new solution-local RCL and then referencing the new RCL in a Blazor WebAssembly project causes an exception at build time, the details of which are below.

This is consistently reproducible in the latest nightly build.

NOT: It also happens if you're targeting a .NET 8.0 project with the .NET 9.0 SDK. This means if you're trying to upgrade a solution to .NET 9.0, setting the target framework to net8.0 does not solve the problem.

/cc @javiercn

Expected Behavior

I expect everything to compile just like it did in .NET 8.0.

I would also expect that the unit tests for this new SDK could be complete enough to have caught this already. It seems like whomever is working on this feature is not doing enough real-world testing... it's really late in the game to still be running into these types of issues.

Steps To Reproduce

1. Create a new Blazor WebAssembly Standalone project on .NET 9.0.
2. Add a new .NET 9.0 Class Library to the project.
3. Change the Class Library to be a Microsoft.NET.Sdk.Razor project.
4. Paste the following code into the Class Library project file:

   <ItemGroup>
       <PackageReference Include="Z.Blazor.Diagrams" Version="3.*" />
       <PackageReference Include="Z.Blazor.Diagrams.Algorithms" Version="3.*" />
   </ItemGroup>

5. Right click the "Dependencies" node of the Blazor project and select "Add Project Reference".
6. Check the box for the Class Library and save.
7. Compile.

Exceptions (if any)

Conflicting assets with the same target path '_content/Z.Blazor.Diagrams/default.styles.min.css.gz'. For 'All' assets 'Identity: C:\Users\Me.nuget\packages\z.blazor.diagrams\3.0.2\staticwebassets\default.styles.min.css.gz, SourceType: Package, SourceId: Z.Blazor.Diagrams, ContentRoot: C:\Users\Me.nuget\packages\z.blazor.diagrams\3.0.2\staticwebassets\, BasePath: _content/Z.Blazor.Diagrams, RelativePath: default.styles.min.css.gz, AssetKind: All, AssetMode: All, AssetRole: Primary, AssetRole: , AssetRole: , RelatedAsset: , AssetTraitName: , AssetTraitValue: , Fingerprint: cxftpe9ktm, Integrity: IMm7Tn1WUfqPdT8luDMhqPlEI1m8fYwxUlQ5yKtfsl4=, CopyToOutputDirectory: Never, CopyToPublishDirectory: PreserveNewest, OriginalItemSpec: C:\Users\Me.nuget\packages\z.blazor.diagrams\3.0.2\staticwebassets\default.styles.min.css.gz' and 'Identity: C:\Users\Me\source\repos\RC2 Blazor Problem\ClassLibrary1\obj\Debug\net9.0\compressed\qzawkj1i64-6pwzqlbbfs.gz, SourceType: Package, SourceId: Z.Blazor.Diagrams, ContentRoot: C:\Users\Me\source\repos\RC2 Blazor Problem\ClassLibrary1\obj\Debug\net9.0\compressed\, BasePath: _content/Z.Blazor.Diagrams, RelativePath: default.styles.min.css.gz, AssetKind: All, AssetMode: All, AssetRole: Alternative, AssetRole: , AssetRole: , RelatedAsset: C:\Users\Me.nuget\packages\z.blazor.diagrams\3.0.2\staticwebassets\default.styles.min.css, AssetTraitName: Content-Encoding, AssetTraitValue: gzip, Fingerprint: r2ddnv2k36, Integrity: jQmpYy2v5ZLjHZv9IylfmhJjj5D93cygKxHrQWiyS68=, CopyToOutputDirectory: Never, CopyToPublishDirectory: PreserveNewest, OriginalItemSpec: C:\Users\Me.nuget\packages\z.blazor.diagrams\3.0.2\staticwebassets\default.styles.min.css'.

.NET Version

9.0.100-rc.2.24423.15

Anything else?

Here is a project that reproduces the issue: RC2 Blazor Problem.zip

[javiercn]

@robertmclaws thanks for contacting us.

The issue is happening because the package ships with their own pre-compressed assets.

Which you can see in https://nuget.info/packages/Z.Blazor.Diagrams/3.0.2 and which conflicts with the assets that we generate.

Removing the pre-compressed assets, and letting the system handle the compression, makes the problem go away. Alternatively, compression can be disabled for those particular assets.

    <Target Name="RemoveConflictingAssets" BeforeTargets="ResolveStaticWebAssetsConfiguration">
        <ItemGroup>
            <StaticWebAsset Remove="@(StaticWebAsset)" Condition="'%(Extension)' == '.gz'" />
        </ItemGroup>
    </Target>

I expect everything to compile just like it did in .NET 8.0.

That's our aim.

I would also expect that the unit tests for this new SDK could be complete enough to have caught this already. It seems like whomever is working on this feature is not doing enough real-world testing... it's really late in the game to still be running into these types of issues.

Our aim is to catch issues as soon as we can, and we have a comprehensive set of tests for this feature. You are welcome to check on the SDK repo. We do have compatibility testing that goes and upgrades existing apps; however, I don't think is realistic to expect that all bugs are caught before they reach users. We can always do things to improve, but that's not going to change the fact that sometimes we will miss some things.

In this case, the assets present in the package are incompatible with the compression feature because it expects the assets to be placed on the same location on disk, which is what the error is pointing out.

The metadata in the package is malformed because the compressed assets don't contain the metadata that is used to identify such compressed assets, hence why the system doesn't detect them.

In this case the package should not ship the pre-compressed versions on the package or should do so including the appropriate metadata.

I've filed https://github.com/dotnet/aspnetcore/issues/57518 to track a concrete improvement for this case.

[robertmclaws]

 <Target Name="RemoveConflictingAssets" BeforeTargets="ResolveStaticWebAssetsConfiguration">
    <ItemGroup>
        <StaticWebAsset Remove="@(StaticWebAsset)" Condition="'%(Extension)' == '.gz'" />
    </ItemGroup>
</Target>

Thanks for this workaround! I was able to get my project compiling with it.

To be clear for anyone reading, this is supposed to go in the Blazor WebAssembly Client app (and Server app, if it exists), and not the RCL.

Our aim is to catch issues as soon as we can, and we have a comprehensive set of tests for this feature.

You're awesome and I'm not trying to beat you up over it. From what I understand from other tickets, a whole set of tests for this SDK involving WebAssembly Classic were missed. I'm just trying to point out as gently as possible that there have been pretty basic blockers that seem like they could have been thought through and tested.

And I do so because it seems like the Blazor compilation system keeps getting re-written, so for .NET 10 I would hope the team would not make the same mistakes in overlooking WebAssembly Classic.

Looking at the project I referenced there does not appear to be anything custom happening that would have "mangled" the metadata.

So, if the package is "malformed", it appears to be the result of standard compilation operation of the previous release, and it would likely be expected that every RCL out there currently has the same "mangled" metadata.

So then, if you import any existing <= .NET 8 RCL into a .NET 9 RCL, you can expect the same behavior in a WebAssembly client & server project.

[javiercn]

@robertmclaws it's the fact that the package is producing gzip files and putting them in the package without integrating with static web assets.

This is the first package that I've seen in the wild do this, and I don't believe it's intentional, as there's additional work needed at runtime to serve the gzipped versions. I suspect the tool generates compressed versions out of the box, and those simply get included in the package. That happens to be incompatible with the built-in compression we now provide out of the box.

As for the webassembly bit, I want to make sure that we set the record straight. Preview releases are not subject to the same standards of quality as RTM releases or Release Candidates. In the case that you mentioned with regards to webassembly, we actually caught the regression before the release, we just didn't have time to fix it without having someone work on the weekend, and while it affected a subset of scenarios, the majority of scenarios as confirmed by our tests worked just fine. The issue didn't make it into preview7 but was addressed soon after and available on nightly builds.

I want to make sure that we set expectations correctly. We do our best for all scenarios to work on previews (and obviously on final releases), however, every time an issue comes in, we first evaluate the following (among other things)

• Does it have a workaround?
• How hard it is to apply?
• How many users do we think are impacted by it?
• How risky is the fix?

And based on this we make decisions. Fortunately, with MSBuild, even if an issue breaks building a project, there's normally some MSBuild we can concoct to address the situation. We understand that this is not always ideal, but that's what previews are for. Sometimes things are going to have rough edges and is through feedback that we improve them.

Other times, we simply don't run into the scenario until someone comes in and reports an issue, and it typically involves an uncommon setting/config that is hard for us to predict ahead of time. Even when we go out of our way to test these upgrade scenarios against existing codebases.

Keep in mind that even before the feature even sees the light on a preview, it usually has been integrated in ASP.NET Core, runtime, and other big repos, so it gets a decent level of coverage, but there are situations that we simply don't run into.

So, if the package is "malformed", it appears to be the result of standard compilation operation of the previous release, and it would likely be expected that every RCL out there currently has the same "mangled" metadata.

We offer guidance on how to integrate with static web assets that the package is not following, hence it results in the compressed assets being included without the necessary metadata. This is usually something that we work on with other package authors to make sure they follow best practices to avoid these types of problems. Again, preview releases are what give us the opportunity to detect and fix these before the final release.

[robertmclaws]

Thanks for taking the time to help explaining all of that, I really appreciate it. 🤜🏻

Is the issue that his use of Excubo.WebCompiler is "silently" creating compressed assets by default? If so, that makes sense that .NET 8 didn't process that properly, and I apologize for any mischaracterization of the issue.

It appeared to me initially that that task was just compiling all of the CSS together, but a deeper look shows that there are indeed gzipped assets in the wwwroot folder, which is not an approach you folks recommend.

Understanding that existing packages will cause issues with this (and the task you created will work around it when compiling the WebAssembly app), are there ways that .NET 9 could already take those gzipped assets into account if the Blazor.Diagrams project was re-compiled in .NET 9? That way the metadata wouldn't get "mangled", it would just skip compressing the files and use the existing compressed files to generate new metadata for the package?

Thanks!

[martincostello]

Just spotted this issue as a logged an issue over in the dotnet/sdk that sounds a lot like this one: https://github.com/dotnet/sdk/issues/43033

In my case, it looks like it's to do with Humanizer. It works fine with .NET 8.

[perlun]

Seeing this now in a project of ours when trying to compile with .NET 9 Preview 7, but regretfully the workaround from @javiercn that you (@robertmclaws) mention in https://github.com/dotnet/aspnetcore/issues/57512#issuecomment-2310498743 doesn't seem to help in our case. :thinking:

    /home/plundberg/.nuget/packages/microsoft.net.sdk.webassembly.pack/9.0.0-preview.7.24405.7/build/Microsoft.NET.Sdk.WebAssembly.Browser.targets(347,5): error MSB4018: 
      The "DefineStaticWebAssetEndpoints" task failed unexpectedly.
      System.ArgumentException: An item with the same key has already been added. Key: /home/plundberg/git/path/to/project/bin/Debug/net8.0/wwwroot/_framework/System.ServiceModel.Primitives.resources.wasm
         at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
         at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer)
         at Microsoft.AspNetCore.StaticWebAssets.Tasks.DefineStaticWebAssetEndpoints.Execute()
         at Microsoft.Build.BackEnd.TaskExecutionHost.Execute()
         at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(TaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)

(I added the <Target> node at the end of the relevant .csproj files, right before the closing </Project> tag)

We also tried adding the /v:d flag to the dotnet build command to help debug this, but this somehow seems to be broken on our .NET 9 Preview installs; we get absolutely no more logging despite trying to increase the verbosity like this. :thinking: Filed a separate issue about that, https://github.com/dotnet/sdk/issues/43069. If you have any ideas that could help us figure out "what package is causing this", that would be very helpful. :pray:

[LccBuild]

Seeing this now in a project of ours when trying to compile with .NET 9 Preview 7, but regretfully the workaround from @javiercn that you (@robertmclaws) mention in #57512 (comment) doesn't seem to help in our case. 🤔

Same problem here

[robertmclaws]

@perlun @LccBuild Did you move to the latest RC2 nightly build like I mentioned in the first post? The problem is solved in those builds.

[perlun]

@robertmclaws Thanks, that's an interesting suggestion. I dug around and finally found the links to the nightly builds to both .NET 9 and ASP.NET Core 9 rc2 nightlies (https://github.com/dotnet/sdk/blob/main/documentation/package-table.md#table and https://github.com/dotnet/sdk/blob/main/documentation/package-table.md#table + figuring out some URL manually).

With this in place I'm getting another error though:

      Unable to find package Microsoft.NET.Sdk.WebAssembly.Pack with version (>= 9.0.0-rc.2.24428.8)
        - Found 22 version(s) in nuget.org [ Nearest version: 9.0.0-preview.7.24405.7 ]

I read elsewhere that using nightlies requires special NuGet configuration to work. I think for our part, we'll just postpone this until .NET 9 RC2 is released.

[martincostello]

If you add this NuGet.config to the root of your repo, then the packages will restore:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="dotnet9" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9/nuget/v3/index.json" />
    <add key="NuGet" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
  <packageSourceMapping>
    <packageSource key="dotnet9">
      <package pattern="*" />
    </packageSource>
    <packageSource key="NuGet">
      <package pattern="*" />
    </packageSource>
  </packageSourceMapping>
</configuration>

[perlun]

Thanks for the workaround @martincostello. :+1: I tried with the newly released .NET 9 RC1 (not RC2 as previously mentioned) now, and can confirm that the problem has indeed been resolved for us on this version. :heavy_check_mark:

The exact version number on my machine now, where the problem has been resolved: 9.0.0-rc.1.24431.7.