nguerrera
commented
5 years ago Referencing current HEAD in case it moves between now and repro investigation: https://github.com/RssBandit/RssBandit/commit/628bd5ce8bac111f3a9d7000355b8ca70b94b7b2
Open clairernovotny opened 5 years ago
nguerrera
commented
5 years ago Referencing current HEAD in case it moves between now and repro investigation: https://github.com/RssBandit/RssBandit/commit/628bd5ce8bac111f3a9d7000355b8ca70b94b7b2
nguerrera
commented
5 years ago I was confused on the email thread, this isn't conflict resolution as it's a downgrade during restore.
dsplaisted
commented
5 years ago This should be fixed when we do https://github.com/dotnet/cli/issues/10105. Previously I thought that trimming previous NuGet packages would merely be a perf optimization and conflict resolution would handle selecting the right assets. However this demonstrates a case where there is a functional failure if we don't trim previous packages an old bug can resurface with .NET Core 1.x package references when a RID is specified.
@onovotny I believe adding the following PackageReference to RssBandit.csproj should work around the issue:
<PackageReference Include="Microsoft.NETCore.App" Version="2.2.0" ExcludeAssets="all" PrivateAssets="all" />
clairernovotny
commented
5 years ago Do I want version 2.2.0 or 3.0.0?
dsplaisted
commented
5 years ago 2.2.0. We will likely not have this package on NuGet.org for 3.0.0.
jzabroski
commented
5 years ago @dsplaisted The obvious, simpler, solution to this is to implement CleanFull, which we discussed in a separate topic last week as a meta-step I used when upgrading from .NET Core 2.1 to 2.2: https://github.com/dotnet/sdk/issues/3046#issuecomment-477618439
One logical reason could be that I define a "cleanfull" target in all my .NET Core projects that does the following:
- Runs MSBuild Clean target
- Runs custom task to remove all nuget packages
- Runs custom task to remove MSTest results directory
- Runs custom task to remove bin and obj directories
Trimming the NuGet object graph is the "clever" solution, but probably also too clever by half and adds complexity to an already complex sub-system.
I've created a ticket for this suggestion: https://github.com/dotnet/sdk/issues/3067
jzabroski
commented
5 years ago Nevermind. I see that the issue here is that the packagereference is in the csproj. I misunderstood @nguerrera link to the git commit hash - it's the commit that fixes the issue as well as what caused the issue in the first place. I thought @onovotny was saying that he upgraded to 2.0.4 and still got errors due to cached packages still existing in the packages directory.
dsplaisted
commented
5 years ago A simple repro for this is simply a .NET Core console app with a RuntimeIdentifier and a reference to the log4net package:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifier>win-x86</RuntimeIdentifier>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="log4net" Version="2.0.8" />
</ItemGroup>
</Project>The problem is that the log4net package depends on System.Runtime.Serialization.Formatters version 4.3.0, but on the 4.0.x or 4.1.x versions of a lot of other contracts. With a RID specified, RID-specific dependencies get injected which cause package downgrades such as the following:
error NU1605: Detected package downgrade: System.Runtime.InteropServices from 4.3.0 to 4.1.0. Reference the package directly from the project to select a different version. error NU1605: consoletest -> log4net 2.0.8 -> System.Console 4.0.0 -> runtime.win.System.Console 4.3.0 -> System.Runtime.InteropServices (>= 4.3.0) error NU1605: consoletest -> log4net 2.0.8 -> System.Runtime.InteropServices (>= 4.1.0)
To fix this, we may need something like https://github.com/NuGet/Home/issues/7344. We thought an "empty" Microsoft.NETCore.App package could help with this, but because it doesn't have a runtime.json with RID-specific references to the 2.x equivalents of "runtime packs", it didn't fix the issue in the same way that the explicit reference to the 2.2.0 package suggested as a workaround here did.
jzabroski
commented
5 years ago Warning: The value of this comment might be very low.
I'd be careful about over-engineering this. Looking at .NET Core 3.0, I'm already starting to feel like library maintainers will be screwed once more. A few weeks ago I saw Brice Lambson tweeting about how awesome .netstandard2.1 was now that he could write once target everywhere. But since the direction is for .netstandard to be about "API surface area", many of us are upset at the complexity this will introduce in our projects: We see it as going back to the dark ages once more. For example, I can't "just" take the Span type in a new project as a dependency.
To be honest, I don't know what the right solution is and I don't envy the intrepid soul who is tasked with solving the problem, since some of it is idiosyncratic (how binding redirection only works on strong named assemblies and how strong name assemblies came about prior to the advent of the web semver2.0.0 standard and even Java's OSGI standards).
For me, I am filing this case away as interesting and something to noodle on.
dsplaisted
commented
5 years ago @jzabroski The complexity in this issue is coming from trying to make things simpler and better ("less over-engineered") in .NET Core 3.0, while still supporting packages that were created back in .NET Core 1.x when .NET Core was represented by a complex graph of NuGet packages.
jzabroski
commented
5 years ago I may be wrong, but I think the "trivially hard" case is the one I run into regularly:
--assemblies parameter that loads assemblies that implements an ITask interface exposed by the TaskRunnerDirtyAssemblyResolveHelper for a reason.In other words, while you may solve it for the framework (which is clever), how will you solve it for the more common scenarios? If you think that's not a big problem, consider the most common TaskRunner is Json.Net's Deserialize!
If my comments are wrong here, I apologize - but I have spent A LOT of time thinking about this lately.
dsplaisted
commented
5 years ago @jzabroski Exact version matching (and binding redirects to get around it) are required on .NET Framework, but not on .NET Core. So the issues you refer to (which were indeed painful) are mostly solved for .NET Core.
dsplaisted
commented
5 years ago It's also possible to hit this if you end up with dependencies on contracts from different "bands" of .NET Core 1.x. log4net happens to have these dependencies directly, but it could happen with other combinations of packages. These references also cause the issue:
<PackageReference Include="System.IO.FileSystem" Version="4.0.1" />
<PackageReference Include="System.Reflection" Version="4.3.0" />
A better workaround than adding a reference to Microsoft.NETCore.App is probably to reference Microsoft.NETCore.Targets:
<PackageReference Include="Microsoft.NETCore.Targets" Version="2.1.0" ExcludeAssets="all" PrivateAssets="all" />Is there an ETA on a "transparent" / silent / built-in fix for this? Still happens with preview 6.
ericstj
commented
5 years ago +1 on this, still getting customer reports of this issue. I thought we agreed to fix this by bringing in the empty packages? Now I see you've linked this to a NuGet feature to dummy out packages. Would love that feature, don't get me wrong, but we asked for the same thing in 2.0 and didn't get it (and thus we have conflict resolution). What's your plan to solve this problem in 3.0. This is a regression from 2.1.
nguerrera
commented
5 years ago This is a regression from 2.1.
It is worth noting that the experience regresses when retargeting to 3.0, but does not break without project changes. We hold this distinction in mind strongly. We do reserve the right to require additional steps upon retargeting. Just like the runtime does not guarantee that apps will work 100% of the time on roll forward or retargeting.
I thought we agreed to fix this by bringing in the empty packages?
That doesn't turn out to work. We have to know when to pull in the empty package, which we tried to do based on whether there are any package references, but it turns out that we would have to wait for restore to know if there are any because the only package references could be transitive through a project reference. We went through about 3 different iterations of workarounds and none of them worked. At some point, we have to cut our losses. The workaround can be applied manually.
If we pulled in the empty package all the time, we'd break offline build of FDD with only framework references, which is a key value prop. Or we'd have to go back to having offline nuget sources/fallback folders as part of the SDK, which was a huge burden and also something we're very intent on keeping out from 3.0 onwards.
we asked for the same thing in 2.0 and didn't get it (and thus we have conflict resolution).
We are actively avoiding this strategy. We deliberately worked closely with NuGet for FrameworkReference instead of working around limitations. Conflict resolution added quite a maintenance burden.
The NuGet fix for this was on the list of requests from the beginning of the FrameworkReference discussions with NuGet going back about a year. NuGet has avoided doing this particular work item in 3.0 on the assumption that this does not impact mainline scenarios. Prior to that, there was a mistaken assumption that the NuGet fix would only cause perf penalty of leaving more work to conflict resolution.
There was some data gathering about which packages in all of nuget.org would cause a conflict like this due to mismatched 1.x packages in the graph, and I believe it was deemed tolerable for 3.0 given that data and given that a workaround exists. If there are data points that contradict this, then we need the details on that. And we should share that data with the NuGet team.
For the most part, it is believed that there is a better workaround of upgrading components to get 1.x era packages out of your graph.
In the specific case of System.Device.Gpio, it is netstandard2.0, but requires transitive 1.x dependencies. Can we fix that?
I would also push on log4net to have a clean netstandard2.0 TFM without dependencies on 1.x packages. This adds value irrespective of the fix here. The performance of the build will be better to name one.
nguerrera
commented
5 years ago cc @rrelyea @nkolev92
eerhardt
commented
5 years ago I hit this today upgrading ML.NET to 3.0. I was able to distill it down to this simple repro. Restoring this project:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.11.3" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
</ItemGroup>
</Project>with the 3.0.100-preview8-013437 SDK results in:
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: Detected package downgrade: Microsoft.Win32.Primitives from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version.
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> System.Net.Primitives 4.0.11 -> runtime.win.System.Net.Primitives 4.3.0 -> Microsoft.Win32.Primitives (>= 4.3.0)
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> Microsoft.Win32.Primitives (>= 4.0.1)
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: Detected package downgrade: System.Net.Primitives from 4.3.0 to 4.0.11. Reference the package directly from the project to select a different version.
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Net.Primitives (>= 4.3.0)
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> System.Net.Primitives (>= 4.0.11)
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: Detected package downgrade: Microsoft.Win32.Primitives from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version.
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> System.Net.Sockets 4.1.0 -> runtime.win.System.Net.Sockets 4.3.0 -> System.Security.Principal.Windows 4.3.0 -> Microsoft.Win32.Primitives (>= 4.3.0)
F:\DotNetTest\ThreeOhBug\ThreeOhBug.csproj : error NU1605: ThreeOhBug -> Microsoft.NET.Test.Sdk 15.8.0 -> Microsoft.TestPlatform.TestHost 15.8.0 -> Microsoft.TestPlatform.ObjectModel 15.8.0 -> NETStandard.Library 1.6.0 -> Microsoft.Win32.Primitives (>= 4.0.1)If I literally change any line in the .csproj, it works:
I think many people are going to run into this issue and we should strongly consider getting some mitigation in place.
nguerrera
commented
5 years ago We need https://github.com/NuGet/Home/issues/7344 to fix this. As I said, there has been some data gathering suggesting that could be deferred based on this being not common.
We included that feature request from the beginning of the FrameworkReference work. We tried some workarounds in the SDK and none of them work.
cc @rrelyea The recent uptick in reports on this seems to go against the data gathering that suggested this wouldn't be common.
jods4
commented
4 years ago Hit this today on 3.0-rc1
jzabroski
commented
4 years ago @nguerrera It's hard as an outsider to help fix the problems without knowledge of the user stories and test cases. Do you have failing test cases and a general suite of tests with an Xunit tag around this feature that need to pass to consider this "feature complete"?
I think, given the number of false-starts here and misjudge of complexity, test-driven development and getting the tests to fail first and then write the code might work better.
nguerrera
commented
4 years ago @jzabroski This issue is hard as an insider too. It's simply hard.
If this issue is important to you, the best way to help get it fully resolved is to engage on https://github.com/NuGet/Home/issues/7344. That is the correct fix.
jzabroski
commented
4 years ago Respectfully, that seems down in the weeds. To read https://github.com/NuGet/Home/issues/7344 I need to then also read about https://github.com/dotnet/cli/issues/10006, which proposes breaking up .NET meta-packages into "targeting packs" and "run-time packs". But that issue doesn't clearly define what these two words mean. I had to jump from https://github.com/dotnet/cli/issues/10006 to https://github.com/dotnet/cli/issues/10086 to https://github.com/dotnet/cli/issues/10007 to figure out what a "runtime pack" was, and I'm left understanding:
User stories would be:
As a library author, I would like to take Span as a dependency without targeting .NET Core 3.0, so that I can support .NET Core 1.x applications until end-of-life support finishes.
As a library author, old .NET Core 1.x csproj files referencing "targeting packs" explicitly should elide those dependencies during build, pack and publish. NuGet.org packages should just have a reference to the package.
In this sense, a "targeting pack" is basically a .NET Core 3 plug-in which takes a nuget package reference and installs an object graph (.deps.json) and its concrete assets into the lib directory.
That to me seems like the easiest fix here...
jzabroski
commented
4 years ago tl;dr: I think there is duplicate/cross-effort here between what you are doing and the CLR team designing roll forward functionality. Simplify things by inverting your dependencies and treating targeting packs as plug-ins. I hate to over-simplify and think I solved all your problems in 30 minutes*, but this is my gut reaction to all this design effort.
nguerrera
commented
4 years ago The design for runtime packs and targeting packs is covered by dotnet/designs#50. It is already implemented. NuGet/Home#7344 is about a very specific piece allowing a way to ask nuget to exclude packages that are overridden by platform. It is not as simple as you suggest because there is a deliberate requirement for "targeting packs" to not be so coupled to nuget.
jzabroski
commented
4 years ago @nguerrera Thanks, I read through your dotnet Design Doc, but toward the bottom of the pull request review, I see your caveat lector that says:
There are some things that are out of date. It's on my list to clean it up.
Given .NET Core 3.0 is live, can you please update this PR and fix the "out-of-date" issues?
IDK as a .NET developer how I am expected to understand how the .NET world works when there is no explanation of it in a single place.
ericstj
commented
4 years ago dotnet/extensions just hit a similar problem today. NETStandard.Library 1.6.0 is brought in transitively which brings in 1.x era packages, which bring in the 1.x era private native shims. Due to name changes we renamed these, but still understand the old names (without lib prefix). It'd be nice to have an implicit NETStandard.Library 2.0.3 reference, similar to the Microsoft.NETCore.Platforms reference.
cdavernas
commented
2 years ago Im having a similar issue today. A simple repro is to create a .NET 6.0 project and add a reference to latest CloudNative.CloudEvents package.
Debug and publishing goes as expected for me (on win 11 with VS2022), but as soon as I try to build the auto-scaffolded Dockerfile, I get dozens of downgrade messages due to:
... -> Microsoft.Extensions.DependencyModel 2.1.0 -> Microsoft.DotNet.PlatformAbstractions 2.1.0 -> System.IO.FileSystem 4.0.1 -> runtime.win.System.IO.FileSystem 4.3.0
VS
... -> Microsoft.Extensions.DependencyModel 2.1.0 -> Microsoft.DotNet.PlatformAbstractions 2.1.0 -> System.IO.FileSystem 4.0.1 -> System.IO.FileSystem.Primitives (>= 4.0.1)
Please help!
dsplaisted
commented
2 years ago @cdavernas Have you tried applying the workaround:
<PackageReference Include="Microsoft.NETCore.Targets" Version="2.1.0" ExcludeAssets="all" PrivateAssets="all" />
noseratio
commented
1 year ago Having the same issue with .NET 7.0 and Serilog: https://stackoverflow.com/q/74453395/1768303
This solves the problem. Is there a better solution? Should Microsoft.NETCore.Targets be resurrected?
<PackageReference Include="Microsoft.NETCore.Platforms" Version="7.0.0" />
<PackageReference Include="Microsoft.NETCore.Targets" Version="5.0.0" PrivateAssets="All" />I am not even sure how exactly this fix works. Can some please shed some light on it?
jzabroski
commented
1 year ago I am not even sure how exactly this fix works.
Well, Microsoft.NETCore.Targets has been almost empty since 5.0.0, hence why it was removed later.
You can use Nuget.org Package Explorer to see its contents here: https://nuget.info/packages/Microsoft.NETCore.Targets/5.0.0
It has a lib folder, which contains a netstandard1.0 folder. It used to then contain a runtimes.json file in that folder that stated the runtimes the library targeting netstandard1.0 could run against.
Can some please shed some light on it?
I can't speak to why Microsoft stopped publishing Microsoft.NETCore.Targets, or why your particular solution works here.
There is no one, good explanation of how to manage dependencies in .NET. The whole system is just at least as bad as any other major ecosystem out there.
A good introduction to versioning limitations in .NET is Jon Skeet's blog post of the same name
<MajorVersion>.0.0.0 to prevent runtime minor version assembly load conflicts across "diamond dependencies". EFCore, Newtonsoft.Json and other major libraries follow this approach. This is NOT the same as the nuget package version. So, sometimes when people talk about versioning limitations, they are only referring to assembly load conflicts, and other times they are referring to linking and loading a package written for one targetFramework against another.Another good talk on versioning is the contrarian viewpoint by Rich Hickey, titled Spec-ulation
Another good perspective is what Google does in Bazel/Blaze build system. All transitive dependencies that are actual dependencies need to be explicit.
Java developers further improve this situation with BND, which allows specifying co-constraints on versions so that versions are resolved through constraint satisfaction.
noseratio
commented
1 year ago Thank you @jzabroski, very helpful. On a side note, microsoft.netcore.targets.3.1.0.nupkg is pretty much the same as v5.0.0.
SDK version: 3.0.100-preview4-010963
repro:
EDIT by @dsplaisted: This is the workaround for this bug: