Using central package management causes build breaking NU1009 with MAUI projects

hypdeb commented 1 year ago

Description

Building a MAUI project using central package management does not seem possible because of the NU1009 error. It reads as a warning, but is in fact a build breaking error.

I attached a minimal reproduction containing a MAUI project and a library that this project depends on, which itself requires the ILogger abstraction from Microsoft.Extensions.Logging.Abstractions,. If you build it as is, the build fails with:

error NU1009: The packages Microsoft.Extensions.Logging.Abstractions;Microsoft.Extensions.Logging.Abstractions;Microsoft.Extensions.Logging.Abstractions;Microsoft.Extensions.Logging.Abstractions 
 are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs

If you fix this error by removing the reference to Microsoft.Extensions.Logging.Abstractions (what the error suggests you do), the build fails with a collection of errors, due to the fact that the ILogger interface is cannot be found anymore in the library which the MAUI project depends on.

I also tried using https://learn.microsoft.com/en-gb/dotnet/core/project-sdk/msbuild-props#disableimplicitframeworkreferences, but it seems it is ignored altogether and has no effect.

Moreover, it doesn't seem to happen with other project types such as web API or console app.

Steps to Reproduce

Clone the attached reproduction project repository
Run dotnet build at the root of the cloned repo

Link to public reproduction project repository

https://github.com/hypdeb/repros

Version with bug

7.0 (current)

Last version that worked well

Unknown/Other

Affected platforms

Windows

Affected platform versions

This is a build failure

Did you find any workaround?

No response

Relevant log output

No response

jfversluis commented 1 year ago

@Eilon would you know more about this? What area this might fall on? Feels like this is maybe not something for our team?

Eilon commented 1 year ago

I've heard of this feature but I've never directly used it. I think we'll need a NuGet or SDK expert to chime in. @eerhardt - perhaps you or someone you know could help shed light on this?

eerhardt commented 1 year ago

cc @jeffkl @dtivel @joelverhagen - any ideas/thoughts here?

hypdeb commented 1 year ago

I'm more than happy to report this / have it moved to another repo if this is not the right place for it. I put it here because I can only reproduce it with MAUI projects.

ghost commented 1 year ago

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

hypdeb commented 1 year ago

Should I understand that I have to give up either on MAUI or central package management for the foreseeable future?

Could you perhaps point me in the general direction of the component that is emitting that error s.t. I can try and fix it myself?

Eilon commented 1 year ago

Should I understand that I have to give up either on MAUI or central package management for the foreseeable future?

Could you perhaps point me in the general direction of the component that is emitting that error s.t. I can try and fix it myself?

Not giving up, we're just finding someone who can help route this issue to the right location.

Hopefully one of @jeffkl @dtivel @joelverhagen can help us figure out what the root issue is.

jeffkl commented 1 year ago

The Maui SDK injects package references for you implicitly: https://github.com/dotnet/maui/blob/main/src/Workload/Microsoft.Maui.Sdk/Sdk/BundledVersions.in.targets#L70

When you manage your package versions centrally, NuGet does not want you to define versions for implicitly defined package references since an SDK is trying to manage it for you.

In order to have a <PackageVersion /> in your Directory.Packages.props for a package but not get this error, you'll need to condition it so that when building a Maui project it is not defined:

<Project>
  <PropertyGroup>
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="7.0.0" Condition="'$(UseMaui)' != 'true'" />
  </ItemGroup>
</Project>

The $(UseMaui)' != 'true' condition here disables the <PackageVersion /> item for Maui projects and fixes the NU1009 error.

hypdeb commented 1 year ago

Thanks for the workaround @jeffkl, I will try it later today.

I'm just wondering whether implicitly injected package references are a thing that is desirable. Or at least, shouldn't they be overridable by the user? I would understand if this could trigger a warning, which I would then be able to suppress in case I'm aware that I'm doing this overriding, but I don't understand why it's an error and is preventing me from building.

hypdeb commented 1 year ago

I added the condition and it seems like it is not working. It cannot find the types and extension provided in the logging package.

See commit above.

hypdeb commented 1 year ago

Note that if you build only the Lib project like so

dotnet build .\Repro.Lib\Repro.Lib.csproj

it builds fine.

hypdeb commented 1 year ago

Any other idea I could try out ? Otherwise I will disable my MAUI project for now s.t. I can continue working on my codebase. It's annoying but I cannot figure out a better solution for now, sadly.

jeffkl commented 1 year ago

Which types are not found? I saw that the AddDebug() method wasn't showing but its in a package that is not being referenced. I added this to Repro.Maui.csproj:

<PackageReference Include="Microsoft.Extensions.Logging.Debug" />

And this to Directory.Packages.props:

<PackageVersion Include="Microsoft.Extensions.Logging.Debug" Version="7.0.0" />

And now its working.

What exact types are not being resolved? Some screenshots or logs would be very helpful.

hypdeb commented 1 year ago

Ah damn, you're right. Somehow I thought all the logging related types were missing but it's only this AddDebug. Sorry about the confusion.

In the end, is that the approved way of using central package management with MAUI projects, or do you think this should be improved (e.g. as I suggested above by making it only a warning)? It doesn't feel right that I would have to modify my package versions file depending on the type of projects in my solution.

jeffkl commented 1 year ago

The best fix would be for the Maui SDK to set the IsImplicitlyDefined metadata on any <PackageReference /> items it adds. This allows other parts of the system like NuGet's central package management to know that a particular PackageReference was not added by the user so it can treat it differently.

<ItemGroup>
    <!-- Added by an SDK with IsImplicitlyDefined=true so that other parts of the .NET developer stack know it wasn't added by the user -->
    <PackageReference Include="Some.Package" Version="1.0.0" IsImplicitlyDefined="true" />
</ItemGroup>

MartyIX commented 1 year ago

I'm also affected by the issue. It's great that it looks like the solution is not that hard.

nkolev92 commented 1 year ago

I have some details in https://github.com/NuGet/Home/issues/12443#issuecomment-1439257823, which is the NuGet Client equivalent for this issue.

I think addressing the scenario will require coordination among Maui & NuGet.

dansiegel commented 1 year ago

This issue is popping up all over with MAUI right now. I have hit this with Prism and for an update on Mobile.BuildTools. When I add a project to provide an extension for the MauiAppBuilder this fails with NU1009 because Mobile.BuildTools.Configuration.Maui has a ProjectReference on Mobile.BuildTools.Configuration which has the PackageReference on Microsoft.Extensions.Configuration which is Implicitly Defined by MAUI. This occurs even though Mobile.BuildTools.Configuration.Maui with the extension only has a single project reference and no direct NuGet references.

hypdeb commented 1 year ago

With .NET8 it seems the problem has gotten worse. Another error appears when trying to build a MAUI project within a solution using central package version management:

Severity    Code    Description Project File    Line    Suppression State
Error   NU1008  Projects that use central package version management should not define the version on the PackageReference items but on the PackageVersion items: Microsoft.Windows.SDK.BuildTools;Microsoft.Maui.Controls;Microsoft.Maui.Controls.Compatibility;Microsoft.AspNetCore.Components.WebView.Maui

MartyIX commented 1 year ago

I was forced to revert a PR that introduced CPM in my project. It was too painful to work with. :-|

mnmr commented 1 year ago

This is just a sequence of train wrecks and I'm not seeing a single Microsoft person step in and take charge to fix it.

Someone thought is was a great idea to tie SDK versions to specific versions of Microsoft.Extensions packages..
Then someone figured that it should absolutely be a show-stopping compile error if there is any reference to the package..
No one wants to claim responsibility, so fix gets postponed and MAUI is dead in the water for a lot of folks

Seriously? Flutter, here I come.

hypdeb commented 1 year ago

This is just a sequence of train wrecks and I'm not seeing a single Microsoft person step in and take charge to fix it.

Someone thought is was a great idea to tie SDK versions to specific versions of Microsoft.Extensions packages..

Then someone figured that it should absolutely be a show-stopping compile error if there is any reference to the package..

No one wants to claim responsibility, so fix gets postponed and MAUI is dead in the water for a lot of folks

Seriously? Flutter, here I come.

Although I understand your frustration, as I am also suffering from it, I think this is not so constructive. To me it seems like a simple coordination issue between the people who introduced central package management and the team behind MAUI.

@jeffkl as I understand you are from the team behind CPM. Do you know who on the MAUI team could help ? Maybe @davidortinau (got your name from the blog post announcing MAUI in .NET7) ? I'm willing to do the changes and create the PR, I just need some guidance as I am familiar with neither of the code bases ;(

MartyIX commented 1 year ago

To me it seems like a simple coordination issue between the people who introduced central package management and the team behind MAUI.

The word "simple" made me smile :) You are absolutely right but the issue is resources. People are busy with long backlogs and there are many bugs in MAUI that need fixing. So they are fixed based on perceived priorities. So imho the correct way to be constructive here is to argue that this issue is a blocker for many existing projects (my honest opinion is that it's not but over time it might get worse; still I'm affected too and wishing this gets fixed).

I'm willing to do the changes and create the PR, I just need some guidance as I am familiar with neither of the code bases ;(

Yes, but note that there is also: https://github.com/dotnet/maui/pull/14383/files

So one can be only patient or ... attempt to fix it proactively and be lucky that the fix is deemed correct :)

hypdeb commented 1 year ago

Maybe 'simple' was not the right word. I meant that it's probably not as dramatic as @mnmr suggests.

hypdeb commented 1 year ago

Could we at least revert the change that made it worse in the preview versions of .NET 8 ? This way at least we can get it to work using the workaround above.

@mattleibow It seems it could be due to commit 2dcc1482 where you introduced MauiImplicitPackageReference.

mattleibow commented 1 year ago

Could we at least revert the change that made it worse in the preview versions of .NET 8 ? This way at least we can get it to work using the workaround above.

@mattleibow It seems it could be due to commit 2dcc1482 where you introduced MauiImplicitPackageReference.

We probably can't revert as we actually want to only use these nugets, but you can set <UseMauiNuGets>true and will not import any nugets at all. You then have full control. In another PR https://github.com/dotnet/maui/pull/14576 I fix a bug after the IDE changed things, I actually rename that confusing property (add a new one) to now be EnableMauiImplicitPackageReferences

hypdeb commented 1 year ago

Sorry I'm a bit confused.

You mean I should use <UseMauiNuGets>false</UseMauiNuGets> as a property in my MAUI project file to not import any of the NuGet packages implicitly referenced by MAUI right ?

And you're saying that you have renamed it to EnableMauiImplicitPackageReferences ? Should I then switch to using EnableMauiImplicitPackageReferences once your PR is merged ?

mattleibow commented 1 year ago

UseMauiNuGets is working today. I am adding a second property once my PR is merged, but UseMauiNuGets will still work.

I just don't want to promote the UseMauiNuGets property just yet as it is a bad name and may go away. It was more for internal use so I never gave much thought to the name.

TL;DR is UseMauiNuGets will work and should work in net7 and net8.

Syed-RI commented 1 year ago

@mattleibow just to clarify, I can.use <UseMauiNuGets>true and then let's say reference a nuger package of Maui i.e 7.0.52 as a workaround to pin that version for my project regardless of which workload is installed?

En3Tho commented 1 year ago

Just hit that after migration to CPM. My project is a blazor monorepo with multiple hosting models in one solution. Everthing except Maui works correctly. This is so frustrating because: 1) This requires hacks to be added to cpm file and otherwise build just fails 2) There are warnings about how I should not include reference because it is implicitly referenced 3) And when I don't reference it explicitly my build just fails because it detects a package downgrade because maui project references another with higher version

This is so weird. So much ceremony. And especially saddening when it's just a Logging package. Come on...

juwens commented 1 year ago

Just hit that after migration to CPM. My project is a blazor monorepo with multiple hosting models in one solution. Everthing except Maui works correctly. This is so frustrating because:

This requires hacks to be added to cpm file and otherwise build just fails

There are warnings about how I should not include reference because it is implicitly referenced

And when I don't reference it explicitly my build just fails because it detects a package downgrade because maui project references another with higher version

This is so weird. So much ceremony. And especially saddening when it's just a Logging package. Come on...

They only develop MAUI, they don't need to use it, let alone try it with more than one project in their solution. So they easily can make big claims on how AWSOME and production ready it is. Just listen to one of their community standups or .net conf talks. It's sickening.

But don't worry, all problems will be fixed with the put_next_major_version_here release, and it will be so AwSoMe!!

mattleibow commented 1 year ago

Did you try maui for dotnet 8 yet? Does it work for you?

bwhittaker69 commented 1 year ago

Can someone please explain this more simply? This is very confusing I have no idea where any of these elements being referred to go ... which file? which directory? which project? .... its very frustrating.

bwhittaker69 commented 1 year ago

Which types are not found? I saw that the AddDebug() method wasn't showing but its in a package that is not being referenced. I added this to Repro.Maui.csproj:

What exact types are not being resolved? Some screenshots or logs would be very helpful.

When I search through all files in any of my projects or solutions I can find no references anywhere to these elements.

Cheesebaron commented 1 year ago

This issue makes it impossible to use <UseMauiEssentials> in a library with <ManagePackageVersionsCentrally>. Please fix it!

Why throw errors just because I happen to use something that MAUI Essentials pulls in?

/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.TeamSports/TrackMan.AppCore.TeamSports.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Microsoft.Maui.Graphics are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]
/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore/TrackMan.AppCore.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Microsoft.Maui.Graphics;Microsoft.Maui.Graphics;Xamarin.Android.Glide are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]
/Users/tomaszcielecki/git/AppCore/Playground/Playground.Droid/Playground.Droid.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Xamarin.Android.Glide;Xamarin.Google.Android.Material are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]

En3Tho commented 1 year ago

@Cheesebaron see this workaround from @jeffkl : https://github.com/dotnet/maui/issues/12953#issuecomment-1409212256

Cheesebaron commented 1 year ago

@Cheesebaron see this workaround from @jeffkl : #12953 (comment)

Doesn't seem to work for me.

Did this in my Directory.Packages.Props:

diff --git forkSrcPrefix/Directory.Packages.props forkDstPrefix/Directory.Packages.props
index c919d35848db73c4edc8d9f1a75f821f48cfd57d..4f045d5e300d9460f8cfdc112277fe1e0210d847 100644
--- forkSrcPrefix/Directory.Packages.props
+++ forkDstPrefix/Directory.Packages.props
@@ -23,7 +23,7 @@
     <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="7.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Logging.Console" Version="7.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Options" Version="7.0.1" />
-    <PackageVersion Include="Microsoft.Maui.Graphics" Version="6.0.501" />
+    <PackageVersion Include="Microsoft.Maui.Graphics" Version="6.0.501" Condition="'$(UseMaui)' != 'true'" />
     <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.6.0" />
     <PackageVersion Include="Microsoft.SourceLink.AzureRepos.Git" Version="1.1.1" />
     <PackageVersion Include="Microsoft.TestPlatform.ObjectModel" Version="17.0.0" />
@@ -68,10 +68,10 @@
     <PackageVersion Include="xunit" Version="2.4.2" />
     <PackageVersion Include="XunitXml.TestLogger" Version="3.0.78" />
     <PackageVersion Include="ImageCaching.Nuke" Version="3.0.0" />
-    <PackageVersion Include="Xamarin.Android.Glide" Version="4.13.2.2" />
+    <PackageVersion Include="Xamarin.Android.Glide" Version="4.13.2.2" Condition="'$(UseMaui)' != 'true'" />
     <PackageVersion Include="Xamarin.AndroidX.ConstraintLayout" Version="2.1.4.2" />
-    <PackageVersion Include="Xamarin.Google.Android.Material" Version="1.7.0.1" />
+    <PackageVersion Include="Xamarin.Google.Android.Material" Version="1.7.0.1" Condition="'$(UseMaui)' != 'true'" />
     <PackageVersion Include="TrackMan.Design.VirtualGolf" Version="0.1.10" />
     <PackageVersion Include="UXCam" Version="3.2.1" />
   </ItemGroup>
-</Project>
\ No newline at end of file
+</Project>

This still produces the errors:

/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore/TrackMan.AppCore.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Microsoft.Maui.Graphics;Microsoft.Maui.Graphics;Xamarin.Android.Glide are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]
/Users/tomaszcielecki/git/AppCore/Playground/Playground.Droid/Playground.Droid.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Xamarin.Android.Glide;Xamarin.Google.Android.Material are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]
/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.TeamSports/TrackMan.AppCore.TeamSports.csproj : error NU1009: The packages Microsoft.Maui.Graphics;Microsoft.Maui.Graphics are implicitly referenced. You do not typically need to reference them from your project or in your central package versions management file. For more information, see https://aka.ms/sdkimplicitrefs [/Users/tomaszcielecki/git/AppCore/TrackMan.AppCore.sln]

Kebechet commented 1 year ago

Put this directly into your maui .csproj <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" VersionOverride="7.0.1" />

for me this fixed the problem

mattleibow commented 1 year ago

In dotnet 8, we are using nugets for everything, so you don't need to add UseMauiEssentials anymore.

I know this does not fix it for dotnet 7,but there is a bit of difficulty where essentials is a part of the framework, but depends on nugets.

One thing we can do is add a property to allow your projects to skip the nugets and you can then control it.

We actually do that for net8 with $(EnableMauiImplicitPackageReferences)

Cheesebaron commented 1 year ago

Just add net7.0 targets to the Maui Essentials package? Then you don't even need to add any extra props...

mattleibow commented 1 year ago

I think that may be a good idea tbh... I see the Microsoft.Extensions packages have more than one tfm... I am thinking essentials and graphics could fall into this category. The main maui framework is tfm specific, but the extras are extensions... Let me do a PR and see what shakes out. Probably can do the same for resizetizer as well...

Thanks for this idea that I totally should have considered since I did nuget analysis when we transitioned.

mattleibow commented 1 year ago

I am trying here, but alas things are "not so great".

I tried making the builds compile, and it is mostly OK:

But, the usage is very hard. We will first need to enable you to disable the implicit things:

https://github.com/dotnet/maui/pull/17046

Then you could potentially do this:

<ItemGroup>
    <!-- include .NET MAUI -->
    <FrameworkReference Include="Microsoft.Maui.Core" />
    <FrameworkReference Include="Microsoft.Maui.Controls" />

    <!-- include update Essentials/Graphics-->
    <PackageReference Include="Microsoft.Maui.Graphics" Version="8.0.*-*" />
    <PackageReference Include="Microsoft.Maui.Essentials" Version="8.0.*-*" />
</ItemGroup>

<ItemGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android'">
    <!-- include .NET MAUI dependencies -->
    <PackageReference Include="Xamarin.Android.Glide" Version="4.13.2.2" />
    <PackageReference Include="Xamarin.AndroidX.Legacy.Support.V4" Version="1.0.0.15" />
    <PackageReference Include="Xamarin.AndroidX.Lifecycle.LiveData" Version="2.5.1.1" />
    <PackageReference Include="Xamarin.AndroidX.Navigation.Common" Version="2.5.2.1" />
    <PackageReference Include="Xamarin.AndroidX.Navigation.Fragment" Version="2.5.2.1" />
    <PackageReference Include="Xamarin.AndroidX.Navigation.Runtime" Version="2.5.2.1" />
    <PackageReference Include="Xamarin.AndroidX.Navigation.UI" Version="2.5.2.1" />
    <PackageReference Include="Xamarin.Google.Android.Material" Version="1.7.0" />
</ItemGroup>

This feels amazing, finally you have control. Alas! Google has gone and ... well caused immeasurable suffering. First, we can't easily update AppCompat because they went and renamed a type that we are using. Yay! Then also they moved a type from one package to another, so we now have duplicate types???

The best way I can see here is for the .NET 7 branch to update the AndroidX dependencies to the same that .NET 8 branch is using. But now we have gone and updated things so if you build a library using the latest bits, anyone with an older .NET install could get all sorts of issues.

mattleibow commented 1 year ago

I also tried adding .NET 7 to all the packages, but this now causes other issue where .NET packages do not have .NET 7 TFMs:

https://github.com/dotnet/maui/pull/17048

I think the previous comment still stands, we can't do too much magic, we just need to update net7 to use the latest AndroidX and then try and just make Essentials/Graphics work backwards - but that is still scary... But less so than all the other options.

En3Tho commented 1 year ago

@mattleibow Thank you for looking into this. I really hope build-wise MAUI experience will get better (other notable thing is workloads but I've heard there is ongoing work on making those side-by-side available. I really don't get why just we can't use <WorkloadReference Name="tfm blabla" Version="version blabla"/> but whatever).

mattleibow commented 1 year ago

OK, I am not sure this will be possible to make work in net7. The issue is just that AndroidX is totally a free-for-all, so many API changes in minor version bumps. You can see the pain Xamarin.Forms had with "I Updated AndroidX something from 2.1.1 to 2.1.2 and now my app crashes".

Everything can work, except that a minor update to any AndroidX packages requires a rebuild and even a re-working of maui. The "nice" thing about workloads is that now you can't update to break things 😆😭😆😭😆.

With .NET 8 things are no longer workloads so we can potentially look at making .NET 9 things work with .NET 8. For example, a .NET 9 package can have both .NET 8 and .NET 9 TFM. This will mean that we can do "breaking" package updates, and not break people on older things.

Other than that, my PR got merged and probably will go out in the next next version of MAUI: https://github.com/dotnet/maui/pull/17046

This will allow you to disable all the implicit things, and then use the CPM to track it. This does not allow you to update those AndroidX things as it will explode your app.