.NET 6 RC 1: fails to restore FSharp.Core in certain cases

rolfbjarne commented 2 years ago

Issue Description

Create an F# library project.
Create a C# project, and reference the F# library project.
Add the following NuGet.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" protocolVersion="3" />
    <add key="Dotnet arcade" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
    <add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
    <add key="macios-dependencies" value="https://pkgs.dev.azure.com/xamarin/public/_packaging/macios-dependencies/nuget/v3/index.json" />
    <add key="xamarin-impl" value="https://pkgs.dev.azure.com/azure-public/vside/_packaging/xamarin-impl/nuget/v3/index.json" />
  </packageSources>
</configuration>

Make sure to use the latest .NET 6 RC 1 release.
Restoring the C# project now fails because it can't find FSharp.Core.

Steps to Reproduce

Repro: fsharplibrary-a9884c8.zip

Download & extract test case.
Download and install .NET 6.0.100-rc.1.21458.1 (other versions might work, but you'll have to update the included global.json accordingly).
dotnet restore main/*.csproj

Expected Behavior

Successful restore.

Actual Behavior

$ dotnet restore main/*.csproj
  Determining projects to restore...
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102: Unable to find package FSharp.Core with version (>= 6.0.0-beta)
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 40 version(s) in dotnet-public [ Nearest version: 5.0.3-beta.21352.5 ]
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 0 version(s) in /usr/local/share/dotnet/library-packs
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 0 version(s) in xamarin-impl
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 0 version(s) in Dotnet arcade
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 0 version(s) in dotnet6
/Users/rolf/test/dotnet/fsharplibrary/main/main.csproj : error NU1102:   - Found 0 version(s) in macios-dependencies
  Failed to restore /Users/rolf/test/dotnet/fsharplibrary/main/main.csproj (in 1.06 sec).
  Restored /Users/rolf/test/dotnet/fsharplibrary/lib/fsharplibrary.fsproj (in 1.09 sec).

A couple of notes:

Restoring just the F# library project works just fine: dotnet restore lib/*.fsproj
Removing the F# library from the C# project works fine too.
Adding the following to the NuGet.config the test case work:

diff --git a/NuGet.config b/NuGet.config
index 447ea35..507aca0 100644
--- a/NuGet.config
+++ b/NuGet.config
@@ -4,6 +4,7 @@
     <clear />
     <add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" protocolVersion="3" />
     <add key="Dotnet arcade" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
+    <add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />
     <add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
     <add key="macios-dependencies" value="https://pkgs.dev.azure.com/xamarin/public/_packaging/macios-dependencies/nuget/v3/index.json" />
     <add key="xamarin-impl" value="https://pkgs.dev.azure.com/azure-public/vside/_packaging/xamarin-impl/nuget/v3/index.json" />

Versions & Configurations

This happened between 6.0.100-rc.1.21455.2 and 6.0.100-rc.1.21458.1.

Maestro bump that triggered this: https://github.com/xamarin/xamarin-macios/pull/12663/commits/9f8ea0330e8c1cc68a5efafbec9a8e21af392463

Attach a binlog

msbuild.binlog.zip

rainersigwald commented 2 years ago

Isn't that build of the SDK one of the final candidate builds, meaning that it doesn't publish packages to the normal feeds but only to a specific feed? I believe the corresponding one is https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=6.0.100-rc.1.21458.32-shipping (though I'm not quite sure about the least significant version part there; 32 versus 2). I do see a matching FSharp.Core package there: https://dev.azure.com/dnceng/public/_packaging?_a=package&feed=6.0.100-rc.1.21458.32-shipping&package=FSharp.Core&protocolType=NuGet&version=6.0.0-beta.21457.3.

I believe this will be resolved when RC1 is fully released and the packages are available on nuget.org.

rainersigwald commented 2 years ago

@mmitche does that ☝🏻 sound right to you? Should Maestro have flowed a new feed?

rolfbjarne commented 2 years ago

@rainersigwald The fact that the package is found when restoring the F# library project by itself, but not when restoring the C# project that references the F# project, suggest that there's another problem.

rolfbjarne commented 2 years ago

Also if I change the C# project to an F# project (that references another F# project), then the restore works too.

mmitche commented 2 years ago

@mmitche does that ☝🏻 sound right to you? Should Maestro have flowed a new feed?

That would be the case for final stable builds, but not for RC builds. Reason being that final stable builds will try and create the same package over and over again as we iterate on new functionality. We can't and don't want to overwrite packages in a feed, so we have to create a new feed for each new build, and that has to automatically flow using Maestro.

For RC/Preview builds, there is no stabilization, so we publish to the "main" feeds on each build. They key bit here is that tooling (C#, F#, msbuild, nuget, etc.) publish to dotnet-tools and not dotnet6. The reason is that tooling is not necessarily dotnet version specific (especially since a lot of the tooling supports Framework too). So publishing to the dotnet6 feed doesn't really make sense. Instead we use dotnet-tools.

The https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=6.0.100-rc.1.21458.32-shipping feed serves two main purposes:

It's a way to gather all packages that support a given .NET SDK release together in one place for testing. If I were to take this feed and nuget.org and smash them together, representing the final "released" state, the product should work.
It's the home of the signed packages. .NET 6 signs after the build, so the dotnet6 feed is mostly unsigned. When we sign the packages, we end up with a package with different content but the same name+version, which cannot be published back to the dotnet6 feed. So we use this feed as the home of the signed packages.

You don't necessarily need this feed for testing and validation, but it is one option. The other option is to add dotnet-tools.

dotnet / msbuild