search

dotnet / source-build

A repository to track efforts to produce a source tarball of the .NET Core SDK and all its components
MIT License
256 stars 125 forks source link

8.0 VMR all legs failing in runtime with detected package downgrade #4344

Closed Winniexu01 closed 2 weeks ago

Winniexu01 commented 3 weeks ago

Internal/8.0.1xx build: https://dev.azure.com/dnceng/internal/_build/results?buildId=2433696&view=results

/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Collections.Immutable from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> Microsoft.Build.Tasks.Core 17.8.5 -> System.Collections.Immutable (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> System.Collections.Immutable (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605: Warning As Error: Detected package downgrade: Microsoft.Build.Framework from 17.8.5 to 17.8.3. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605:  Microsoft.NET.Sdk.WebAssembly.Pack.Tasks -> Microsoft.Build 17.8.5 -> Microsoft.Build.Framework (>= 17.8.5)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605:  Microsoft.NET.Sdk.WebAssembly.Pack.Tasks -> Microsoft.Build.Framework (>= 17.8.3) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Collections.Immutable from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> Microsoft.Build 17.8.5 -> System.Collections.Immutable (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> System.Collections.Immutable (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Reflection.Metadata from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> Microsoft.Build 17.8.5 -> System.Reflection.Metadata (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> System.Reflection.Metadata (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Reflection.Metadata from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605:  MonoAOTCompiler -> Microsoft.Build 17.8.5 -> System.Reflection.Metadata (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
/vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605:  MonoAOTCompiler -> System.Reflection.Metadata (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

The related SDK files are not generated due to the errors mentioned above, CentOSStream8_Online_CurrentSourceBuiltSdk_x64 and Fedora38_Offline_CurrentSourceBuiltSdk_x64 are failed: Copy Previous Build step:

found 0 files
Finishing: Copy Previous Build

Prep the Build step:

tar (child): /mnt/vss/_work/1/vmr/prereqs/packages/archive/dotnet-sdk-*.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Build step:

Custom SDK '/vmr/.dotnet/dotnet' does not exist or is not executable
dotnet-issue-labeler[bot] commented 3 weeks ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

MichaelSimons commented 3 weeks ago

@NikolaMilosavljevic - Can you jump on this? TIA. This is related to https://github.com/dotnet/runtime/pull/100595

NikolaMilosavljevic commented 3 weeks ago

It did not repro when I built locally with latest changes: https://dev.azure.com/dnceng/internal/_git/dotnet-dotnet/commit/75831229196c2d6d803830ad1558d20e673a16be?refName=refs/heads/release/8.0.1xx

I'm not sure why that build had a different commit that is now missing.

MichaelSimons commented 3 weeks ago

Look at the internal/release/8.0.1xx branch - https://dev.azure.com/dnceng/internal/_git/dotnet-dotnet?path=/&version=GBinternal/release/8.0.1xx&_a=history

NikolaMilosavljevic commented 3 weeks ago

It seems that runtime now gets PVP flow for Microsoft.Build which bumps it to live (n-1) version (17.8.5) which brings 8.0.0 dependencies of System.Reflection.Metadata and System.Collections.Immutable.

17.8.5 was resolved from previously-source-built source.

Runtime defines S.C.I and S.R.M dependencies version as 7.0.0. https://github.com/dotnet/runtime/blob/ca4f0fe37455882baa00c75b1ef30a7ff1494457/eng/Versions.props#L127

    <SystemCollectionsImmutableVersion>7.0.0</SystemCollectionsImmutableVersion>

Same for the other assembly.

lewing commented 3 weeks ago

These are toolset dependencies, the Tasks in question go into workloads and are run in-process with the sdk/VS durring application build

in net9 this is explicitly called out by the properties that are used https://github.com/dotnet/runtime/blob/main/eng/Versions.props#L135-L139

cc @akoeplinger

NikolaMilosavljevic commented 3 weeks ago

Updating runtime, to remove direct package references (S.C.I and S.R.M.) was not enough. MSbuild repo was regressed with the same runtime PR (https://github.com/dotnet/runtime/pull/100595), and downgraded the versions from 17.8.5 to 17.8.3. Runtime PR added msbuild intermediate package reference with SourceBuild tag. That caused VMR sync to pick the old MSBuild SHA, instead of the one specified in sdk repo.

I tried, locally, to create a patch for runtime to remove the intermediate dependency, to check if vmr-sync would do the right thing. It did not update msbuild to a newer SHA. This runtime change is still needed and I will include it in the same patch with other runtime change mentioned above. However, we need to find a way to update VMR. If sync is not going to bring the correct msbuild SHA, can we update it directly?

@MichaelSimons @premun