Package applicability in NuGet package manager - Search & Update by TFM

[anangaur]

Status: Reviewing

The specification is available here: Package applicability in NuGet package manager UI

The prerequisite for this issue is https://github.com/NuGet/NuGetGallery/issues/3098. The implementation will start server side.

Client side discussions should happen on this issue.

This issue covers,

• PM UI applicability.
• add package applicability.
• list package --outdated applicability. See linked issues.

[To help searching for this issue: "Search by TFM" and "Update by TFM"]

[maartenba]

Just some random thoughts having had this thing in a mind background thread.

A first question would be: will this be a server-side or client-side thing? Or, as I would expect, both sides?

Package metadata does not contain the relevant info required to make a decision whether to include/exclude a result, so this should be based on telemetry. Ideally, this telemetry is something that can be collected by all NuGet clients, e.g. NuGet.exe, dotnet.exe, VS, Rider, Paket, ...

Ideally the format of this telemetry is a protocol extension (given this client-side SearchContext, fetch me my results!) where every client can build such a search context and contribute data as it sees fit (e.g. project type, target framework, ...)

The server can then use this context (ideally with a well-known decision tree?) and gather results that best match the query and search context.

And then of course the other one to consider: privacy. What info can be legally shipped to the server to come up with a better result?

[maartenba]

With the spec now announced: see previous comment :-)

[maartenba]

(or: what is the Server-Side spec for this?)

[karann-msft]

@maartenba this spec is focused on the end-user experience. Once we are close to finalizing that, we'll start a discussion around the implementation details.

[maartenba]

Thanks! Looking forward to seeing the technical spec later on.

[anangaur]

@karann-msft there are some asks on having the framework information on gallery too. Can you include that in your spec? If not, create a sibling spec that takes care of the gallery changes and search improvements both on the gallery and on VS UI.

[nkolev92]

Based on the current proposal for GT, it'll need to consider the new package type as well. So tools would not get shown in the PM UI. https://github.com/NuGet/Home/wiki/Global-Tools---NuGet-Implementation Name still a subject to change, but want to make sure it's tracked.

[karann-msft]

@nkolev92 I don't think there would ever be a reason for GT packages to show up in PM UI, correct? Since they are not meant to be installed to project?

[nkolev92]

@karann-msft Yeah, that's what I meant. I've updated the comment to say "Name still a subject to change" cause I assume that was the confusing part?

[karann-msft]

@nkolev92 I will update this spec to call out that Tools packages will never be shown in the PM UI, but this should be tracked as part of the GT spec since GT packages are never applicable or compatible with any project and we should safeguard against it.

[nkolev92]

@karann-msft Yep, that's already called out in the tools spec and implemented as such :)

A similar ask came through https://github.com/NuGet/Home/issues/6484. We should consider the handling of the MSBuildSdk package type in search based on the decisions in that issue.

No "actionables" here yet, before we narrow that down.

[mhutch]

Figured I'd chime in here instead of creating a new issue.

I'd love to have a type parameter on the search and autocomplete web APIs so MSBuild intellisense can provide correctly filtered suggestions for SDKs and tools.

[nkolev92]

Similarly to DotnetTool, when implementing Package Applicability should account for the package types chosen for the runtime/targeting packs. #7337

[anangaur]

Have we decided upon the name of the packageType for these scenarios?

[nkolev92]

@anangaur No, I kicked the tires on that discussion, before commenting. I wanted to make sure that it doesn't slip through the fingers.

[nkolev92]

The package types in question are: DotnetTool and DotnetPlatform.

The Framework Reference feature also introduces other considerations. https://github.com/NuGet/Home/wiki/%5BSpec%5D-FrameworkReference-in-NuGet

[rrelyea]

Note> Update by TFM has a good scenario in AzDO bug 745617 - create a netcore 2.1 project that uses this package: https://www.nuget.org/packages/Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore

You'll be suggested an update (via update tab) to 3.0.0 versions of this package, despite the fact that the TFM in the new package is only netcoreapp3.0

[HolisticDeveloper]

This would be quite useful, especially with the recent release of .NET Core 3.0 packages. The PM UI offers to upgrade the EntityFrameworkCore packages, but they are only compatible with .NET Standard 2.1 while my projects still target .NET Core 2.2.

Additionally, after reviewing the user experience spec, I would offer a couple of additional suggestions for the Updates scenario.

1 - Icon for available, but incompatible, updates

The spec indicates that when opening the PM UI, the number displayed next to Updates should indicate the number of compatible updates available. It would also be useful to have an additional icon if incompatible updates are detected.

2 - Ignore specific package version

A user should be able to ignore a specific available version so that the PM UI doesn't continually suggest updating to it. It would be great if this info could be stored in the project file or otherwise shared with a team. I understand this scenario might be out of the scope of this issue -- perhaps it would be better suited as a separate feature suggestion?

[anangaur]

Great feedback @HolisticDeveloper 👍 As we iterate on the spec and finalize it, these will definitely come in handy.

[joelverhagen]

Better late than never, @mhutch 😄?

Figured I'd chime in here instead of creating a new issue.

I'd love to have a type parameter on the search and autocomplete web APIs so MSBuild intellisense can provide correctly filtered suggestions for SDKs and tools.

We have a package type parameter in the search and autocomplete APIs allowing you to filter to a specific package type, e.g. https://azuresearch-usnc.nuget.org/autocomplete?q=azure&packageType=dotnettool&semVerLevel=2.0.0&prerelease=true

Autocomplete docs Search docs

Package applicability (a.k.a. search by TFM) will likely take package type into account but will require additional parameters from the client, e.g. project framework.

[joemey]

Note> Update by TFM has a good scenario in AzDO bug 745617 - create a netcore 2.1 project that uses this package: https://www.nuget.org/packages/Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore

You'll be suggested an update (via update tab) to 3.0.0 versions of this package, despite the fact that the TFM in the new package is only netcoreapp3.0

This is happening again with the netcore 3.1 -> net 5.0 especially with the ASP.NET packages which are locked to a TFM.

Right now I have to search through the VS nuget client update list for packages I want to update, because I must ignore all the ASP.NET packages which are not compatible.

[bairog]

This is happening again with the netcore 3.1 -> net 5.0 especially with the ASP.NET packages which are locked to a TFM.

Right now I have to search through the VS nuget client update list for packages I want to update, because I must ignore all the ASP.NET packages which are not compatible.

And now it's is happening again with latest ASP .NET Entity Framework Core packages which are locked to a TFM (latest Entity Framework Core 6.0 is available only for net 6.0). Upgrading large solution TFM from net 5.0 to net 6.0 is a real pain, again. @heng-liu @nkolev92 @anangaur Is there any estimated timeframe for implementing this functionality?

[Bellarmine-Head]

It is happening again:

https://github.com/dotnet/aspnetcore/issues/38256

[Bellarmine-Head]

Re. https://github.com/dotnet/aspnetcore/issues/38256 I think people face two problems:-

1) Going for the suggested updates could break their solution. 2) Genuine updates to the 5.0.11 versions of Microsoft.AspNetCore.Components.WebAssembly and similar packages (that could and should be applied) will be hidden by the fact that incompatible/inappropriate v6 versions are being offered.

[nkolev92]

Hey all,

A lot of this functionality will be server dependant. The good news is that we're looking into adding the respective support to nuget.org.

Please take a look at https://github.com/NuGet/Home/pull/11374 and leave us any feedback you may have!

Going for the suggested updates could break their solution.

Just a small clarification, NuGet itself does not suggest/recommend updates. NuGet shows available updates.

[bairog]

@heng-liu @nkolev92 @anangaur Is there any estimated timeframe for finally implementing this functionality? Suggested updates still break solution targeting older TFMs (.netcore 3.1 and .net5.0).

[Bellarmine-Head]

New .NET release (v7), same problem with NuGet...

[Bellarmine-Head]

I had to dig in order to find out that version 6.0.11 even existed for these packages. The 7.0.0 was hiding that fact.

[bairog]

@heng-liu @nkolev92 @anangaur Is there any estimated timeframe for finally implementing this functionality? Suggested updates still break solution targeting older TFMs (.netcore 3.1, .net5.0 and today .net6.0).

[Bellarmine-Head]

It should be noted that .NET 6 is LTS, and supported until November 12, 2024.

.NET Core 3.1 and .NET 5 may legitimately be regarded as old, but .NET 6 certainly is not. So the experience of using it should, ideally, not be degraded in this way.

[MagicAndre1981]

@Bellarmine-Head this can be fixed by supporting allowedVersions like in packages.config. With this we could easily only see 6.0.x updates

[Bellarmine-Head]

Thanks @MagicAndre1981

If you have some suggestions for Microsoft re. an easy fix for this, please do consider adding a comment to my other ticket (on the Microsoft side) for this issue:-

https://github.com/dotnet/aspnetcore/issues/38336

[Bellarmine-Head]

As I said in my other ticket, imagine Windows Update offering you Windows 11 updates for your Windows 10 install... they fail to install and obscure genuine Windows 10 updates that you really need for security. The notion is laughable. Yet that is what is effectively happening here, re. .NET 6 packages.

[jmoralesv]

Is there any update on this issue?

[JonDouglas]

Hi friends,

Small update. While there are general painpoints every major .NET release, we had to do some server work that we shipped recently to start to expand our options of providing a great experience where you get applicable packages on the client side.

We are shipping another parallel effort that may help you fix these inconsistencies(blog coming soon), but largely this specific work stream will require a fresh set of eyes to the existing spec, perhaps a new experience proposed, and simplifying these experiences now that we have made NuGet more "TFM aware".

We understand the current challenges with being prompted updates for new packages targeting incompatible TFMs and it will take some more time to make progress on this front. I know this isn't what you want to hear, but we're almost unblocked on the server side.

[Bellarmine-Head]

Sounds like real progress @JonDouglas - thanks. Please do keep us posted on here, e.g. re your coming blog piece.

[Bellarmine-Head]

A new .NET release... and the same old problem in Visual Studio:-

Here version 8.0.0 hides the fact that I can (and should) upgrade my WebAssembly packages from 7.0.12 to 7.0.14.

It happens every year with depressing regularity and predictability.

[JonDouglas]

Yes it sucks but we are moving in the right direction here. We're working on exposing the TFM APIs to the v3 api which will allow us to do something more here.

[MagicAndre1981]

again, until this larger TFM API issue is resolved give us allowedVersion attribute from packages.config back and we can hide the 8 packages in for example .net 6 branch @JonDouglas

[MagicAndre1981]

any updates on this?