Build for desktop framework on non-windows platforms

[dasMulli]

The dotnet CLI allowed to build net* targets on mac and linux using the reference assemblies installed with mono. It seems this is not supported by the targets in Microsoft.NET.Sdk. For fun, I also tried referencing the targeting pack NuGets which also didn't work.

Is there a plan for bringing back support for building net* projects on linux/mac? Or should it already work?

[nguerrera]

We would like this to work, but haven't had the time yet to invest in it.

I would prefer that msbuild common targets could resolve the mono targeting pack directory, rather than special work in the SDK to locate it.

[eerhardt]

Here's the current code that resolves the Mono reference assemblies:

https://github.com/dotnet/core-setup/blob/master/src/Microsoft.Extensions.DependencyModel/Resolution/DotNetReferenceAssembliesPathResolver.cs

Someone (dotnet/SDK or core MSBuild) could call into this code to get the path. (There is also an environment variable available to override the path).

[nguerrera]

Aside: That class has a super general name, no docs, and only actually finds the directory on non-Windows.

I think there should be a root reference assembly directory resolved by msbuild that works on all platforms, and we should use that throughout. It is super strange to me that when we call this, we then have to handle windows separately: https://github.com/dotnet/sdk/blob/c1a276fb456606e2b19d4646f8239d2546790d42/src/Tasks/Microsoft.NET.Build.Tasks/FrameworkReferenceResolver.cs#L33

This should all be in one place.

[nguerrera]

So it looks like msbuild already has $(TargetFrameworkRootPath) which the user can override (but we wouldn't respect). However, it seems to not set it to non-empty when the task uses a default value.

Here's how I'd like things to work:

1. MSBuild can also find Mono's xbuild-framework dir on non-Windows and use that as $(TargetFrameworkRootPath) without user intervention.
2. MSBuild sets $(TargetFrameworkRootPath) when it uses a default location on any platform.
3. SDK uses $(TargetFrameworkRootPath) anywhere it needs this path outside of what RAR does for us.

@rainersigwald @jeffkl What do you think?

[eerhardt]

Agreed that ALL the logic should exist in one place.

Another aside here is that this code needs to run/work at runtime of the app because things like ASP.NET MVC Razor View compilation needs to find these assemblies.

That's the reason at least part of this code (the non-windows part) is contained in DependencyModel - which is used both at build and run time.

[nguerrera]

If msbuild can safely take a dependency on the same thing that finds it at runtime, then we can truly have only one place.

Regardless of duplication, though, everything running in the build to resolve this directory the same way GetReferenceAssemblyPaths does. Otherwise, there's a mismatch where RAR can respect $(TargetFrameworkRootPath) set by user, but other code in the SDK cannot, etc.

[eerhardt]

Maybe the most appropriate thing to do here is for the tooling (MSBuild, SDK, etc) to write the location into the .deps.json file during "build" so it can be used by "run". The DependencyModel can use that value as one more place to look for ref assemblies.

During "publish" the value wouldn't get written into the .deps.json file because the ref assemblies get copied to the "refs" folder. Thus a published app doesn't need to get the location Reference Assemlies folder, since it won't exist on the end-users machine.

[nguerrera]

@eerhardt That sounds like the right place to land indeed.

[dasMulli]

Wouldn't it be simpler - from a end-user perspective - to pull the complete set of reference assemblies as a NuGet package? Similar to referencing Microsoft.NETCore.App.. dotnet run and dotnet test would still have to figure out where the mono executable is but just building an application would not rely on a system-wide install of another component. That way using a docker container with just the CLI installed would be enough to produce a runnable output.

[nguerrera]

We have talked informally about having classic targeting packs available as nuget packages before. @terrajobst, any plans there?

[mellinoe]

We've had https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.TargetingPack.NETFramework.v4.6.2 (and other .NET framework versions) for a while, but they're only used for internal builds and aren't published to nuget.org. I think they contain the same assemblies as the real targeting pack.

[rainersigwald]

@mellinoe IIRC I tried to use those (for a different, internal project) and they were missing some key parts.

I once talked about this offline with @akoeplinger, but didn't reach a conclusion.

MSBuild on .NET Core doesn't support finding references in all the same ways we do on Mono/Desktop framework. Most relevant is probably GAC resolution, but targeting packs are also important. As mentioned above, those should be nugetizable--in which case you could probably get stuff built pretty well. There are some other task differences though (GenerateResources springs to mind with the different possibilities of embedding for .NET Core and desktop).

The deps-file approach for run does seem reasonable to me too (given the current CLI run behavior).

[nguerrera]

I don't care at all about GAC resolution. If your build depends on it, it's busted as far as I'm concerned. We never should have gone to the GAC for compilation references. I've removed GAC from the default RAR search paths in the SDK.

Targeting packs via nuget would certainly be nice, but do we need to block on it? What's the harm in probing for xbuild-frameworks?

As far as other differences (GenerateResources), we should distinguish between what are temporary gaps and what is fundamentally never going to work on Core.

[borgdylan]

I hope that this gets resolved by RTM. Currently this makes the csproj variant of the .NET CLI a non-starter for most of my projects.

[gulbanana]

this feature would allow roundtripping projects between vs and vs4mac. right now, you can't do that if they have TargetFrameworks like net461;netcoreapp1.0

[borgdylan]

Has this been worked on for preview 4 of the CLI?

[borgdylan]

After checking I confirm that this is still an issue with preview 4. Are there any known workarounds?

[nguerrera]

I'm taking a look at doing something simple to unblock this.

[borgdylan]

Thanks, even passing in some magic property would help.

[nguerrera]

So as a workaround, explicitly passing/setting TargetFrameworkRootPath with path to xbuild-frameworks (e.g. /p:TargetFrameworkRootPath=/usr/lib/mono/xbuild-frameworks) should work, but doesn't.

The issue is that the mono redist list files have a redirect of the TargetFrameworkDirectory to somewhere else. msbuild understands this redirect, but currently only if it's actually running on Mono:

https://github.com/Microsoft/msbuild/blob/cb8c727a898e4439a3cd18f329504e436b0b7c00/src/Utilities/ToolLocationHelper.cs#L3106-L3111

:(

[borgdylan]

I did try that workaround and it did not work as you mentioned. Mono happens to redirect 4.6.x down to 4.5 to the 4.0 directory for example. It also does not list all the assemblies, relying on the automatic selection of all assemblies in the specified directory instead.

[borgdylan]

Can we open an issue against msbuild to have the behaviour change? Or have they mentioned that this will not change?

[alexvaluyskiy]

Any progress here?

[borgdylan]

I asked on the msbuild repo but no one answered so far.

[dasMulli]

fyi I've had success using the targeting pack nuget (adding the dotnet-core feed to my NuGet.Config). Not using this for production though but.. works on my machine ™️ This works with the latest source build of the cli:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>netcoreapp1.0;net461</TargetFrameworks>
    <RuntimeIdentifiers>osx.10.11-x64;win7-x64</RuntimeIdentifiers>
  </PropertyGroup>

  <PropertyGroup Condition=" '$(TargetFramework)' == 'net461' ">
    <RuntimeIdentifier>win7-x64</RuntimeIdentifier>
    <FrameworkPathOverride>$(NuGetPackageRoot)microsoft.targetingpack.netframework.v4.6.1\1.0.1\lib\net461\</FrameworkPathOverride>
  </PropertyGroup>

  <ItemGroup Condition=" '$(TargetFramework)' == 'net461' ">
    <PackageReference Include="Microsoft.TargetingPack.NETFramework.v4.6.1" Version="1.0.1" ExcludeAssets="All" PrivateAssets="All" />
  </ItemGroup>
</Project>

[DustinCampbell]

I just tried migrating OmniSharp to .csproj and this is a super-painful problem. First of all, there's no guarantee of where Mono might be installed to set the $(TargetFrameworkRootPath). For example, Mono might be in different locations on OSX depending on whether it was installed via Homebrew or through the Mono pkg.

[borgdylan]

The old CLI superseded that issue with environment variables. People may still have those set. The solution is to use those.

[DustinCampbell]

that's not a great solution. This is a major regression for existing customers. We should raise this issue up.

[borgdylan]

But the same environment variable had been working since the KRE and DNX days.

[DustinCampbell]

Yes, but MSBuild has changed a lot recently in support of Mono. However, many of those changes only work when MSBuild is actually running on Mono via runtime checks. I looked at this today and found that the tasks will override any environment variables with incorrect values. Setting TargetFrameworkRootPath is simply not enough without running MSBuild on Mono.

The high-order bit here is that cross-gen is broken for .NET Framework projects on all platforms but Windows.

[borgdylan]

I heard in another github issue, that the CLI will check for MSBuild compiled for .NET proper on Windows so that that version gets used instead of the bundled .NET Core version of MSBuild. Could that happen for mono as well? The mono team has been mentioning for a long time that they want xbuild out and MSBUild running on mono in the mono distribution itself. People like me that need to target mono would have the MSBuild that runs on mono installed with mono itself when that becomes real.

[dasMulli]

I believe there are a few architectural issues to decide on related to MSBuild and Mono:

1. Should MSBuild be able to cross-compile from (mono, full, core) surface / runtime to any other platform? (I'm in favour but that would need some rework from MSBuild afaik)
2. Should the CLI or MSBuild figure out what components are installed locally? (e.g. find mono)
3. Should Mono ship a version of MSBuild with SDKs installed that should be used for compiling and running multi-targeting apps? If so, should it have its own CLI similar to dotnet or integrate with the existing CLI by making it forward to mono's MSBuild?
4. Should there be public targeting pack NuGet packages that enable compiling out of the box? (probably useless since you typically would want to run tests during a build as well..)

[DustinCampbell]

IMO, #1 is infeasible without the CLI running MSBuild on Mono. This is what I ended up doing for OmniSharp in order to handle .NET Framework TFMs and .NET Core TFMs in a single solution.

Other possible mitigations:

• Should 'dotnet build' succeed for the "buildable" projects (e.g. those projects targeting netcoreappX.X or netstandardX.X)? In such cases, I would assume that we'd also display a warning to let the user know that not everything was compiled.
• 'dotnet build' should provide error messages with better guidance. The error message that is currently displayed to the user tells them to go install a targeting pack, which really isn't all that helpful on a Mac. :smile:

[DustinCampbell]

I may take a look at creating a "mono-build" CLI verb at some point in the near future. For OmniSharp, I'll either need to do this or move it off of the CLI entirely. This is a blocker for that project.

[borgdylan]

ping

[alexvaluyskiy]

@dasMulli Is your workaround works fine in dotnet CLI rc3?

[dasMulli]

@alexvaluyskiy haven't had any problems yet..

[alexvaluyskiy]

I've got the error CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/home/alexvaluyskiy/MonoTest/MonoTest.csproj]

[dasMulli]

@alexvaluyskiy sry I completely forgot that macOS is now case insensitive by default.. changed the path in my hack to be lowercase since NuGet works around case sensitivity by lowercasing. (just gave it a try with a docker container)

[alexvaluyskiy]

@dasMulli Thanks, it works for me too. But it is not really a workaround. For example, this code could be compiled only on Mono+net46, but not on Mono+net45 (Mono bug)

IPAddress address = IPAddress.Parse("127.0.0.1");
var newAddressIPv4 = address.MapToIPv4();
var newAddressIPv6 = address.MapToIPv6();

But I can build it with the targeting packs

[borgdylan]

This issue has been assigned a milestone, but what will the resolution be? Will we get the mono variant of MSBuild or will the sdk infrasture change how i functions to all targeting desktop profiles unconditionally?

[TheAngryByrd]

I just updated https://github.com/dotnet/netcorecli-fsc/wiki/.NET-Core-SDK-rc4#using-net-framework-as-targets-framework-the-osxunix-build-fails to show how to use set FrameworkPathOverride as an environment variable if you already have mono installed so you don't have to grab Microsoft.TargetingPack.NETFramework.v4.6.1 package

[StefH]

I have the same issue when building my solution in AppVeyor when referencing .NETFramework,Version=v2.0.

C:\projects\simmetrics-net\.dotnetcli\sdk\1.0.1\Microsoft.Common.CurrentVersion.targets(1111,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v2.0" were not found.

More details at https://ci.appveyor.com/project/StefH/simmetrics-net/build/1.0.2.5

[jtattermusch]

FYI, this is also blocking gRPC C# from migrating to the new .csproj project format https://github.com/grpc/grpc/issues/10441

[DustinCampbell]

I solved this issue for OmniSharp by using the msbuild that ships with Mono 4.8.0 after setting a few environment variables to point to the .NET Core SDK: https://github.com/OmniSharp/omnisharp-roslyn/blob/dev/msbuild.sh. Note: On Linux, this requires installing the "msbuild" package as well: https://github.com/OmniSharp/omnisharp-roslyn/blob/dev/BUILD.md#linux.

[dasMulli]

Also, it looks like a future mono msbuild will carry the SDKs: https://github.com/mono/msbuild/tree/xplat-master/sdks

[DustinCampbell]

Yup. That's correct.

[dasMulli]

So the mono 5 beta is able to build SDK projects using its version of msbuild (which is option nr 3 from one of my above comments). I feel like this issue can be closed since there is nothing to do on the SDK side, the remaining work is probably UX only - like making the CLI forward to / use the system msbuild (https://github.com/dotnet/cli/issues/5723) and making the vstest targets run on mono. Any objections / remaining work?

[dasMulli]

Closing since resolving reference assemblies is now clearly a responsibility of the hosting MSBuild distribution.

[dsplaisted]

We would still like for the CLI to be able to build for .NET Framework out of the box. To do this we are thinking of shipping the reference assemblies as NuGet packages.