clarify versions to use for netstandard libraries

ctaggart commented 5 years ago

This is to update Notes and Guidance on FSharp.Core https://fsharp.github.io/2015/04/18/fsharp-core-notes.html

Is this change correct? It looks to me like the nuget packages 4.3.x are the first to support netstandard2.0 and 4.2x are the first to support netstandard1.6. Should libraries target the highest or lowest third digit? For example, for a library targeting netstandard2.0, should it target FSharp.Core nupkg or 4.3.1 or 4.3.4?

Trying to figure out what https://github.com/xyncro/chiron/pull/100 should be set to. cc @ninjarobot

ctaggart commented 5 years ago

I think if you are targeting netstandard2.0, you want to set the FSharp.Core nuget to 4.3.4. That will avoid this warning:

warning NU1605: Detected package downgrade: FSharp.Core from 4.5.2 to 4.3.4. Reference the package directly from the project to select a different version.

Which you currently get with Chiron 6.3.0 because it depends on FSharp.Core nuget 4.5.2:

dotnet new console -lang F# -n testChiron
cd testChiron/
dotnet add package Chiron

Separately, shouldn't a template for an application have the latest FSharp.Core:

<PackageReference Update="FSharp.Core" Version="4.5.2" />

cartermp commented 5 years ago

I don't think this is the correct change, since the package (starting with 4.2.x) is built against netstandard1.6 and is thus fully compatible with anything that implements that standard version. It also has a netstandard2.0 group in the package so that you can avoid additional downloads when restoring.

This whole article probably needs to be rewritten. There is a lot of useful information if you are on older versions of F# and .NET Framework, but things are a lot simpler even as of a few years ago. For example, the section about deploying FSharp.Core with your app has been default behavior for a while, yet its placement and guidance as-written implies you will have extra deployment concerns for newer applications.

ninjarobot commented 5 years ago

Is FSharp.Core 4.2.3 the appropriate minimum version to target for a netstandard2.0 library? Is this guidance still valid (seems more recent than much of the article)?

As of February 2018 new editions of F# libraries should generally do the following:

Be netstandard1.6, or netstandard2.0 with an additional .NET Framework 4.5 (net45) build Use FSharp.Core nuget 4.2.3 (assembly version 4.4.1.0) or 4.3.3 (assembly version 4.4.3.0)

cartermp commented 5 years ago

It is, but I think every library should be targeting 4.5.2 right now if they want to also work with .NET Core and don't have compat concerns for much older F# consumers:

Fully binary compatible, so F# 4.1 users can use it with no issue.
Targets the same surface area, since the base dependency is still netstandard1.6.

So there doesn't seem to be much to gain by targeting a lower version. The guidance in the article is dated in that sense, because it's valid for much older versions of F# and .NET Framework, but not really valid for .NET Core/.NET Standard. That said, it will become valid again once we change the base dependency in 2019. That's when .NET Core 1.x and .NET Standard 1.x are officially unsupported. We'll target the lowest possible .NET Standard version that is still supported at that time.

At any rate, these nuances should probably be documented more clearly in this document. That would likely have to come from a rewrite.

ninjarobot commented 5 years ago

Depending on the SDK, you'll get warnings about the downgrade from 4.5.2 to an older version, though. And yes, people can fix it by explicitly referencing 4.5.2, but targeting a lower version means it's up to the library consumer to decide, rather than the library forcing them.

cartermp commented 5 years ago

Correct, though I'd argue that's not really a bad thing because it keeps people upgrading, and it's easy to suppress if you're still in the process of upgrading.

It's the (probably) age-old issue of how much library developers should make concessions to people who either don't upgrade or only partially upgrade (e.g., upgrade all dependencies ASAP, but not the .NET SDK they use). Compatibility is a legitimate concession to make, but I don't think I'd do the same just to avoid people getting warnings when they update their package dependency but not the rest of their system to deal with that dependency.

At any rate, if your aim is to use the lowest possible FSharp.Core that will work with .NET Standard 1.6 or higher, then 4.2.3 would be the one to use. That should cover people who haven't updated their SDK since over a year ago - these people are small in number, but they likely do exist.

ctaggart commented 5 years ago

I don't think this is the correct change, since the package (starting with 4.2.x) is built against netstandard1.6 and is thus fully compatible with anything that implements that standard version. It also has a netstandard2.0 group in the package so that you can avoid additional downloads when restoring.

@cartermp This is not correct. 4.3.x is the first to include netstandard2.0 group in the package so that you can avoid additional downloads when restoring.

4.2.3

https://www.nuget.org/packages/FSharp.Core/4.2.3

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
  <metadata>
    <id>FSharp.Core</id>
    <version>4.2.3</version>
    <title>FSharp.Core for F# 4.1</title>
    <authors>Microsoft and F# Software Foundation</authors>
    <owners>Microsoft and F# Software Foundation</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <licenseUrl>https://github.com/Microsoft/visualfsharp/blob/master/License.txt</licenseUrl>
    <projectUrl>https://github.com/Microsoft/visualfsharp</projectUrl>
    <description>FSharp.Core redistributables from Visual F# 4.1.0
                Supported Platforms:
                    .NET 4.5+           (net45)
                    netstandard1.6      (netstandard1.6)</description>
    <summary>FSharp.Core for F# 4.1</summary>
    <language>en-US</language>
    <tags>Visual F# Compiler FSharp functional programming</tags>
    <dependencies>
      <group targetFramework=".NETStandard1.6">
        <dependency id="System.Collections" version="4.0.11" />
        <dependency id="System.Console" version="4.0.0" />
        <dependency id="System.Diagnostics.Debug" version="4.0.11" />
        <dependency id="System.Diagnostics.Tools" version="4.0.1" />
        <dependency id="System.Globalization" version="4.0.11" />
        <dependency id="System.IO" version="4.1.0" />
        <dependency id="System.Linq" version="4.1.0" />
        <dependency id="System.Linq.Expressions" version="4.1.0" />
        <dependency id="System.Linq.Queryable" version="4.0.1" />
        <dependency id="System.Net.Requests" version="4.0.11" />
        <dependency id="System.Reflection" version="4.1.0" />
        <dependency id="System.Reflection.Extensions" version="4.0.1" />
        <dependency id="System.Resources.ResourceManager" version="4.0.1" />
        <dependency id="System.Runtime" version="4.1.0" />
        <dependency id="System.Runtime.Extensions" version="4.1.0" />
        <dependency id="System.Runtime.Numerics" version="4.0.1" />
        <dependency id="System.Text.RegularExpressions" version="4.1.0" />
        <dependency id="System.Threading" version="4.0.11" />
        <dependency id="System.Threading.Tasks" version="4.0.11" />
        <dependency id="System.Threading.Tasks.Parallel" version="4.0.1" />
        <dependency id="System.Threading.Thread" version="4.0.0" />
        <dependency id="System.Threading.ThreadPool" version="4.0.10" />
        <dependency id="System.Threading.Timer" version="4.0.1" />
      </group>
      <group targetFramework=".NETFramework4.5" />
    </dependencies>
  </metadata>
</package>

4.3.4

https://www.nuget.org/packages/FSharp.Core/4.3.4

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
  <metadata>
    <id>FSharp.Core</id>
    <version>4.3.4</version>
    <title>FSharp.Core for F# 4.1</title>
    <authors>Microsoft and F# Software Foundation</authors>
    <owners>Microsoft and F# Software Foundation</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <licenseUrl>https://github.com/Microsoft/visualfsharp/blob/master/License.txt</licenseUrl>
    <projectUrl>https://github.com/Microsoft/visualfsharp</projectUrl>
    <description>FSharp.Core redistributables from Visual F# Tools version 10.1 For F# 4.1
                Supported Platforms:
                    .NET 4.5+           (net45)
                    netstandard1.6      (netstandard1.6)
                    netstandard2.0      (netstandard2.0)</description>
    <summary>FSharp.Core for F# 4.1</summary>
    <language>en-US</language>
    <tags>Visual F# Compiler FSharp functional programming</tags>
    <dependencies>
      <group targetFramework=".NETStandard1.6">
        <dependency id="System.Collections" version="4.0.11" />
        <dependency id="System.Console" version="4.0.0" />
        <dependency id="System.Diagnostics.Debug" version="4.0.11" />
        <dependency id="System.Diagnostics.Tools" version="4.0.1" />
        <dependency id="System.Globalization" version="4.0.11" />
        <dependency id="System.IO" version="4.1.0" />
        <dependency id="System.Linq" version="4.1.0" />
        <dependency id="System.Linq.Expressions" version="4.1.0" />
        <dependency id="System.Linq.Queryable" version="4.0.1" />
        <dependency id="System.Net.Requests" version="4.0.11" />
        <dependency id="System.Reflection" version="4.1.0" />
        <dependency id="System.Reflection.Extensions" version="4.0.1" />
        <dependency id="System.Resources.ResourceManager" version="4.0.1" />
        <dependency id="System.Runtime" version="4.1.0" />
        <dependency id="System.Runtime.Extensions" version="4.1.0" />
        <dependency id="System.Runtime.Numerics" version="4.0.1" />
        <dependency id="System.Text.RegularExpressions" version="4.1.0" />
        <dependency id="System.Threading" version="4.0.11" />
        <dependency id="System.Threading.Tasks" version="4.0.11" />
        <dependency id="System.Threading.Tasks.Parallel" version="4.0.1" />
        <dependency id="System.Threading.Thread" version="4.0.0" />
        <dependency id="System.Threading.ThreadPool" version="4.0.10" />
        <dependency id="System.Threading.Timer" version="4.0.1" />
      </group>
      <group targetFramework=".NETStandard2.0" />
      <group targetFramework=".NETFramework4.5" />
    </dependencies>
  </metadata>
</package>

cartermp commented 5 years ago

Sorry, you're right. Thought it's still fully compatible.

ctaggart commented 5 years ago

I looked up the FShapr.Core implicit versions for each dotnet 2.1 sdk:

I think FSharp.Core 4.3.4 is the best option. It avoids the additional netstandard1.6 dependencies when you are targeting netstandard2.0 and the downgrade warning for dotnet 2.1.30x versions.

ctaggart commented 5 years ago

Closing this issue and hoping @cartermp writes an updated guide.

dsyme commented 5 years ago

@cartermp I checked through the guide up to "Further Technical Notes" and I believe the advice is basically sound. The section @ctaggart was modifying looks like it needs updating.

The thing that should be added is a section explaining implicit SDK references and telling people how to turn them off. Why we need them at all is completely beyond me, they are just needless complexity and should always be turned off.

dsyme commented 5 years ago

It is, but I think every library should be targeting 4.5.2 right now if they want to also work with .NET Core

I agree with @ctaggart that 4.3.4 is the right target version for F# library developers targeting netstandard2.0. As the guidance says, targeting lowest-feasible FSharp.Core versions is generally correct for library developers.

cartermp commented 5 years ago

I still disagree with that. The guidance is about platform support/reach, but FSharp.Core 4.2.3 to FSharp.Core 4.5.2 can be used in the exact same places, since they both package binaries built against the same targets. The only difference is that starting with FSharp.Core 4.3.4, the group in the package manifest lets you use the local nuget cache if you're on an SDK for .NET Core 2.0 or higher.

If people think the guidance should be expanded to include more than platform reach then this is a different question, but I see no reason to target a lower version on this basis.

dsyme commented 5 years ago

@cartermp Pushing up the FSharp.Core reference in a base library like, say, FSharp.Control.AsyncSeq has three effects on the ecosystem

There are some hosts of F# libraries that restrict FSharp.Core, e.g. to 4.4.3.0 or before. By pushing people to FSharp.Core 4.5.0.0, the libraries become silently unusable in these hosts. For example
- All instances of F# Interactive (FSI.EXE) are restricted to some upper FSharp.Core
- iFSharp notebooks processing tools likewise
- Azure Notebooks likewise
- All instances of FAKE.exe likewise including those commonly used for documentation generation tools
- All instances of F# compilation host tooling that hosts type providers likewise
- Azure functions v1 was also in this category.
You force the next level of libraries to make FSharp.Core upgrades. That's mainly bad because of (1)
You force applications to update their FSharp.Core. That is not necessarily such a bad thing but may require re-testing.

(1) is the main reason why silent upgrades of FSharp.Core are so problematic. If we had no library consumption points like the above then it would be different - but we do and they are important to the ecosystem. It is reasonable to expect that over time these library consumption points will slowly raise their FSharp.Core, but it is generally out of control of the user to do that. (It is also reasonable to assume that all these will one day somehow use some kind of mix of project files and pre-configuration but again we are along way from that)

ninjarobot commented 5 years ago

Having a library depend on a newer version of FSharp.Core only to encourage an SDK upgrade is problematic for a few reasons:

unless someone is on the latest SDK, they'll get warnings that don't matter to them. The library works fine with older FSharp.Core. Maybe the developer will upgrade their SDK or maybe they decide to ignore the downgrade errors. Those same "meaningless" warnings for one library are actually meaningful with a library that really depends on features in the newer FSharp.Core, so now we are in danger of people ignoring important warnings.
if everyone pushes people to a newer version than necessary, then developers are painted into a corner when they can't upgrade because of some bug in a newer version of some dependency (or core library or SDK), newer IDE needed to get support, etc. It's an unnecessary constraint.
a library really, really shouldn't have a dependency on the SDK in order to be used. Building the library itself is one thing, but building code that depends on that library should work, as best as possible, on an SDK that can target the runtime that the library supports. The common thread between the SDK and the library should only be that they both can target the same runtime.

I also don't really understand why a library's package dependency on FSharp.Core 4.5.2 would ever result in a downgrade to an older version based on the SDK. With any other dependency, it's going to pull in the minimum version specified rather than do this downgrade, so it's really confusing that FSharp.Core would behave differently.

cartermp commented 5 years ago

I'm afraid I don't understand these points, so correct me if I'm wrong here:

I can use components that reference the binary version 4.5.0.0 in FSI, so presumably I could do so in this other environments.
Due to NuGet restore rules, wouldn't consumers be downgraded to what was restricted? At least that's what was observed here. Or does Paket disallow that package from being referenced entirely? I'd argue that's a bug if it's the case, or at least I'd disagree with such a design decision.

I don't see what's bad about applications or downstream libraries updating their FSharp.Core so long as they don't lose their platform reach. It feels like this discussion is about how to prevent a downgrade warning for consumers who have older toolsets but want to upgrade their dependencies. I don't see this as bad, considering that a goal of .NET Core and the SDK is to roll forward. And as mentioned in the VF# thread, there are dials available for people want want to truly pin themselves down.

cartermp commented 5 years ago

@ninjarobot Just to address these:

if everyone pushes people to a newer version than necessary, then developers are painted into a corner when they can't upgrade [...]

It feels like the answer here is for the developer not to upgrade until they are ready to upgrade.

a library really, really shouldn't have a dependency on the SDK in order to be used.

This isn't the case, though. What you described about what libraries should do is exactly what is possible today, is it not? Unless there's a binary breaking change somewhere, in which case that would certainly be fixed.

dsyme commented 5 years ago

I can use components that reference the binary version 4.5.0.0 in FSI, so presumably I could do so in this other environments.

Only if these libraries are used in a suitably up-to-date FSI (like the one in your VS install that you're using to build the library). They will cause failures if you use them in fsi.exe from VS2017 15.6 or before, and certainly fail in FSI.EXE from VS2015. Likewise they will fail on mono unless you happen to be using the latest Mono packages. (FSI does "try anyway" and forces redirects from 4.5.0.0 --> whatever-fsi-is-using, so sometimes it works. But with the latest changes to async code gen for example you get runtime failures.)

Another example is the fsx2html.exe tool used in F# literate script processing. It has a specific upper FSharp.Core. It is currently 4.5.0.0 but only because I updated it recently. Again this limit exists because it is doing reflective script execution.

ninjarobot commented 5 years ago

not to upgrade until they are ready to upgrade

But they are kind of forced to if the guidance for libraries is to use the newest FSharp.Core whether the library uses that feature or not. Either upgrade your SDK so you don't get warnings (and you use newer FSharp.Core implicitly) or explicitly use the newest FSharp.Core. To be totally clear, I'm talking about libraries that don't depend on the newest FSharp.Core features, but target it anyway. An ancillary library is forcing a core library or SDK upgrade. The tail is wagging the dog.

what libraries should do is exactly what is possible today

it's possible, yes, just guidance should remain that libraries target the lowest dependencies they need, rather than the newest FSharp.Core. It's also worth a mention which specific FSharp.Core version makes sense depending on which netstandard the library targets, although better yet would be that the SDK actually makes that choice, knowing that netstandard1.6 libraries should use a minimum FSharp.Core 4.2.3 unless otherwise specified and netstandard2.0 should use 4.3.4. Just making that an official recommendation would go a long way, even if the SDK can't do that automatically.

I primarily want to have the least cruft possible for someone to use a library. I don't want to force them to upgrade things my library doesn't really depend on just to use my library, and I don't want them to start ignoring warnings just to use it either. If the package manager is satisfied that the project's target platform can support the library, that should be enough.

dsyme commented 5 years ago

guidance should remain that libraries target the lowest dependencies they need, rather than the newest FSharp.Core.

This is absolutely the right guidance. The important caveat is "they need" - if there is something in FSharp.Core 4.5.x that a library needs, it should update.

cartermp commented 5 years ago

@dsyme @ninjarobot Fair enough. Though I feel that shortcomings in tools that ultimately depend on a binary being of a specific upper bound is pushing this problem in the tools to library developers, and the resolution there is to upgrade tools (and in some cases this is the official option, such as older VS 2017 version like 15.6 that are no longer supported). But it's understandable that we can't undo the past for these tools and some developers cannot upgrade their tools. I would question the validity of not being able to upgrade tools yet being able to upgrade packages, but that's a different discussion.

Just a point of clarification, there is no reason to use FSharp.Core 4.2.3 given that 4.3.4 also targets .NET Standard 1.6. It's probably best to pretend that these older 4.2.x variants of the library do not exist.

@ninjarobot This point is interesting:

It's also worth a mention which specific FSharp.Core version makes sense depending on which netstandard the library targets, although better yet would be that the SDK actually makes that choice, knowing that netstandard1.6 libraries should use a minimum FSharp.Core 4.2.3 unless otherwise specified and netstandard2.0 should use 4.3.4

I would actually really love to see FSharp.Core be included in the .NET Standard library metapackage, just like the C# and VB runtime .dlls are. (Side note: the VB runtime .dll is much more analogous to our problem with F# and FSharp.Core, since it also contains essential elements of VB programming that you simply cannot do without).

Perhaps I'll pursue that if it's something people would see as valuable. This would be consistent, and allow this concern you mention indeed be properly driven by the SDK such that it was aware of the binary and its target. This also would alleviate weirdness with C# references to the library, since it'd be in the referenced metapackage.

The prohibitive thing here would be older tooling such as what @dsyme mentioned, but I don't think that's something that should hold it back if it's seen as desirable.

fsharp / fsharp.github.io

clarify versions to use for netstandard libraries #89

4.2.3

4.3.4