Cannot build v6.0.100 tag from source

[sec]

While trying to build SDK from v6.0.100 tag, it fails with:

/home/linux/sdk/src/Layout/redist/redist.csproj : error NU1103: Unable to find a stable package Microsoft.Build.NuGetSdkResolver with version (>= 6.0.0) [/home/linux/sdk/sdk.sln]
/home/linux/sdk/src/Layout/redist/redist.csproj : error NU1103:   - Found 72 version(s) in dotnet-tools [ Nearest version: 6.1.0-preview.1.16 ] [/home/linux/sdk/sdk.sln]
/home/linux/sdk/src/Layout/redist/redist.csproj : error NU1103:   - Found 3 version(s) in dotnet-public [ Nearest version: 5.11.0-rc.10 ] [/home/linux/sdk/sdk.sln]
...

Before that, there was v6.0.100-rtm.21527.8 tag, which could be built without issues, but looks like it was removed/renamed?

[joperator]

I have exactly the same problem. It looks like the package version suffix in Version.props was removed with this commit on behalf of @DavidKarlas' commit in #22241. Please push the missing version 6.0.0 of Microsoft.Build.NuGetSdkResolver to the dotnet-tools package source to allow building the .NET SDK from the v6.0.100 tag.

[DavidKarlas]

@mmitche can you help with this?

[mmitche]

@nkolev92 Can you help out with this? Should this have ended up on nuget.org?

[KirillOsenkov]

@jeffkl as well

[nkolev92]

NuGet doesn't publish that package on nuget.org (since it's Microsoft.*).

[nkolev92]

fwiw, I don't think a stable version of that package has ever been uploaded on any feed.

NuGet automatically pushes the prerelease packages only and given that we don't publish it nuget.org, that package wouldn't be there.

cc @zivkan

[nkolev92]

Someone's been publishing rc versions on nuget.org, https://www.nuget.org/packages/Microsoft.Build.NuGetSdkResolver/5.11.0-rc.10, but I don't know who that is.

Someone who's part of the dotnetframework org should be able to tell who did it.

[nkolev92]

I keep adding comments here, but I don't think it's ideal that the package in question has been published to NuGet.org with that specific verison. Usually you'd want all the correct versions of dependencies for a package available on the same feed (especially on nuget.org), but that's not the case right now (as it's not published along side the NuGet packages, along which it's built).

[joperator]

fwiw, I don't think a stable version of that package has ever been uploaded on any feed.

I think you're right, @nkolev92. As @sec already showed, most versions of Microsoft.Build.NuGetSdkResolver are available in dotnet-tools, but these are all prerelease versions. Without a stable version of this package, it won't be possible to build the .NET SDK from the v6.0.100 tag. But I wonder how .NET 6 could have been released last week if the required version of the package doesn't exist in any of the packages sources specified in NuGet.config.

[zivkan]

But I wonder how .NET 6 could have been released last week if the required version of the package doesn't exist in any of the packages sources specified in NuGet.config.

I don't work in the .NET (Core) Engineering (dnceng) team, but since NuGet inserts into the .NET SDK, we've been impacted by changes to the dnceng systems, so I've learned various things through observations.

Firstly, there are some fundamental issues to acknowledge:

• NuGet, for performance reasons, assumes that package-versions are globally unique and immutable. Therefore once a version 6.0.0 package is downloaded and extracted in the global packages folder, NuGet will never again try to download that 6.0.0 version. In effect, you only have a single chance to publish a package to a public feed. If you discover bugs, you must increment the package version. If you don't, any machine that had the old build of the package will have different behavior to other machines which need to download the package, and this seriously impacts the ability to have deterministic builds.
• Software development is hard, and you never really know what commit is going to be released as GA until it actually ships. You may think you're ready for 6.0.0, but in last minute testing, you find a bug that needs to be fixed, but because you version based on marketing needs, you need to re-use the same version that you previously staged as the release candidate.
• When a shipping product (.NET installer) is composed of components built in different repos (NuGet, MSBuild, .NET SDK, Roslyn, dotnet/templating, etc), it's difficult to get a consistent set of SemVer stable packages when you're really-really ready to release.

Dnceng deal with this by a complex build orchistration system that creates a new Azure Artifacts NuGet feed for every build candidate (I'm avoiding the term release candidate, because it's unrelated to the publicly shipped release candidates) that generate SemVer stable packages, build every repo that participates (which does not include NuGet, for historic reasons) one-by-one, in the correct order, publishing their packages to this "single-build" feed, then building the next repo.

NuGet has historically never inserted SemVer stable versions of packages into the .NET SDK, but because of last-minute issues discovered, it was done "manually" this time. So, NuGet's candidate final build's packages were manually pushed to the .NET "single-build" feed. Hence why the official .NET installer/SDK did not have a build problem.

However, since it was a manual process, the SemVer stable packages were not automally flowed to either the dotnet-tools feed or nuget.org.

Good news is that either someone from the dotnet org can manually push Microsoft.Build.NuGetSdkResolver 6.0.0 to nuget.org, or give the NuGet organization account on nuget.org ownership of that package so we can ourselves. Although, I might be able to push the package to dotnet-tools myself.

Going forward, dnceng and NuGet need to work together to figure out how we're going to manage this automatically. I sent an email to kick off the discussions a week or two ago. I'll ping the thread to make sure this progresses and we don't have a similar problem when 6.0.200 is released.

[mmitche]

I think the issue here was mainly confusion. Microsoft.Build.NuGetSdkResolver @ 6.0.0 was produced by the NuGet build, leading to the update in the dotnet/sdk repo to reference that. But it seems like this package should never get a stable version at all, since it's not pushed to any official endpoint.

As mentioned, the package did exist on an internal, isolated feed. I removed the packages off that feed after release day (as would be done automatically) so that the build would pick up the official nuget.org versions. But since not all the packages were pushed (by design), this was in error.

I think I can simply restore that single package to the feed and all will be good. Hang on will try.

[mmitche]

@sec Please try now.

[joperator]

Now it works for me. Thank you very much @mmitche. Thanks also to @zivkan for the explanations.

[sec]

Working also here, thanks. Closing this one.