search

hedgehogqa / fsharp-hedgehog

Release with confidence, state-of-the-art property testing for .NET.
https://hedgehogqa.github.io/fsharp-hedgehog/
Other
272 stars 30 forks source link

Which .NET version(s) should we support? #242

Closed moodmosaic closed 3 years ago

moodmosaic commented 3 years ago

Things are moving fast now in .NET; which version or versions should we target?

Do we even need to target .NET Framework? Perhaps .NET (Core) is enough? We're still major version 0 in terms of flexibility.

/cc @TysonMN @cmeeren

cmeeren commented 3 years ago

I'm no expert, but my rule of thumb is:

TysonMN commented 3 years ago

In general, the typical strategy is to support the oldest possible targets to increase the set of potential consumers. At the same time, multitargeting allows those using newer targets to take advantage of new features, improved performance, and smaller footprints.

Pragmatically, I would drop support for an older target when the maintenance cost of keeping support becomes "too high". I think this typically manifests as an increasing number of (possibly confusing) compiler directives to execute target-specific code or issues with automate builds. (For example, in PR #210, when I added .NET Framework 4.5 as a target, I had intended to also test that same target, but the automated builds were not working, so tests target version 4.6.1 instead.)

moodmosaic commented 3 years ago

Thank you, both @cmeeren and @TysonMN. I'm wondering how many out there haven't switched entirely to .NET (5, or Core Xy).

What if we drop netXyz entirely and just target netstandard2.0. More or less, this direction would be on the same track with F# itself; starting with F# 5, FSharp.Core 5.0.0 and higher now only targets .NET Standard 2.0...

cmeeren commented 3 years ago

I just came across this post. So it seems the following is a good way to go (if dropping support for .NET Framework before 4.7.2/4.6.1):

Though I'm unsure if net5.0 actually gives any benefits over netstandard2.0 if there are no changes to dependencies.

TysonMN commented 3 years ago

If support for .NET Standard 2.0 is kept and support for all versions of .NET Framework is dropped, then this would probably reintroduce bug #199

I do not know how to statically analyze code and conclude that it suffers from that bug. I only know how to confirm that bug by executing some code. Targeting only .NET Standard [1.5, 2.0] is a necessary condition to exhibit that bug. For that reason, FSharp.Core might or might not exhibit a similar bug even though it only targets .NET Standard.

I think Microsoft encoded a "lie" into NuGet package management by allowing .NET Framework [4.6.1, 4.7.2) to consume .NET Standard [1.5, 2.0] in the same way that any programmer encodes a lie by implementing a function/method by throwing a NotImplementedException.

cmeeren commented 3 years ago

Is it necessary to support .NET Framework < 4.7.2? I mean, Hedgehog is only (intended to be) used for test projects, right? Mostly run on dev machines and CI servers, not on customer devices? Shouldn't those projects be fairly simple to migrate to either net472 or netcoreapp3.1?

TysonMN commented 3 years ago

Sometimes a bug only exists for a specific target. This happens relatively often in language-ext because of all the MSIL it generates. As such, it is reasonable for test projects to include every possible target.

I do not think it is necessary to support .NET Framework [4.6.1, 4.7.1), but supporting .NET Standard. [1.5, 2.0] is also partially supporting .NET Framework [4.6.1, 4.7.1). I think we should follow the the robustness principle by being conservative in what we do. In this case, the conservative position would be to either fully support .NET Framework [4.6.1, 4.7.1) or not support it at all.

cmeeren commented 3 years ago

Sometimes a bug only exists for a specific target. This happens relatively often in language-ext because of all the MSIL in generates. As such, it is reasonable for test projects to include every possible target.

Good point.

I summary then, as I understand it:

moodmosaic commented 3 years ago

I guess in about a year from now, then, we can just target net5.0 and that's it. I expect by that time most users will be on .NET 5.

moodmosaic commented 3 years ago

supporting .NET Standard. [1.5, 2.0] is also partially supporting .NET Framework [4.6.1, 4.7.1)

Exactly... There's the robustness principle on one hand and explicit is better than implicit on the other hand. At some point, perhaps we just target net5.0 and be (hypothetically) 'done'. :thinking:

TysonMN commented 3 years ago

Ah, yes. I also like the "explicit is better than implicit" principle. It definitely applies here. That idea was only in my mind, but I couldn't think of how to concisely convey it. I need to remember that it appears on that list of Python aphorisms.

cmeeren commented 3 years ago

I guess in about a year from now, then, we can just target net5.0 and that's it. I expect by that time most users will be on .NET 5.

I'd like to contest this. There are many users with legacy applications that are not easily updated from .NET Framework. I am responsible for such an app at work. My app targets net472, and depends on lots of legacy code still targeting as low as .NET Framework 2 (yes, not a typo). I'd need netstandard2.0 to test this app using Hedgehog (which I don't anyway).

If it doesn't cost anything to support .NET Framework other than including net461 or netstandard2.0 in the list of targets, I see little reason to get rid of it. If it starts causing us pain on the other hand, then by all means, drop support for them.

moodmosaic commented 3 years ago

Yep, fair enough 👍 Sounds like a plan, for now, then. Let's keep the target(s) as-is and we can come back to this in a year or so.

moodmosaic commented 3 years ago

Thanks for providing such a great feedback, @TysonMN and @cmeeren.

moodmosaic commented 3 years ago

...with that being said, here's what I get on a brand new machine without Visual Studio 201x installed:

$ dotnet build
Microsoft (R) Build Engine version 16.7.0+7fb82e5b2 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Hedgehog -> C:\Snapshot\src\fsharp-hedgehog\src\Hedgehog\bin\Debug\netstandard1.6\Hedgehog.dll
  Hedgehog -> C:\Snapshot\src\fsharp-hedgehog\src\Hedgehog\bin\Debug\netstandard2.0\Hedgehog.dll
C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.5 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\src\Hedgehog\Hedgehog.fsproj]
  Hedgehog.CSharp.Tests -> C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.CSharp.Tests\bin\Debug\netcoreapp3.0\Hedgehog.CSharp.Tests.dll
C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.6.1 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.CSharp.Tests\Hedgehog.CSharp.Tests.csproj]
  Hedgehog.Tests -> C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.Tests\bin\Debug\netcoreapp3.0\Hedgehog.Tests.dll
C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.6.1 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.Tests\Hedgehog.Tests.fsproj]
  Hedgehog.Benchmarks -> C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.Benchmarks\bin\Debug\netcoreapp3.0\Hedgehog.Benchmarks.dll

Build FAILED.

C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.5 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\src\Hedgehog\Hedgehog.fsproj]
C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.6.1 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.CSharp.Tests\Hedgehog.CSharp.Tests.csproj]
C:\Users\Nikos\.dotnet\sdk\3.1.403\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v4.6.1 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks [C:\Snapshot\src\fsharp-hedgehog\tests\Hedgehog.Tests\Hedgehog.Tests.fsproj]
    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:01.05

Even after installing v4.5 and v4.6.1 developer packs...

TysonMN commented 3 years ago

Let's gather some intel. Can you follow these steps?

  1. In the Windows Settings (Windows key -> Settings), click Apps.
  2. In the search box, enter .net.
  3. Take screenshots to show us the results.
moodmosaic commented 3 years ago

Here's what I have, but I think it looks like that after installing v4.7.2 developer pack. Hmm...

C:\Snapshot>reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP"

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\CDF
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4.0

C:\Snapshot>reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\full" /v version

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\full
    version    REG_SZ    4.7.03062

C:\Snapshot>
TysonMN commented 3 years ago

Do you only have command line access to this machine?

moodmosaic commented 3 years ago

Yes, I'm using a DaaS on AWS and from what I see it's running Windows Server 2016. — I have this (only local) workaround for now:

diff --git a/src/Hedgehog/Hedgehog.fsproj b/src/Hedgehog/Hedgehog.fsproj
index 6bc928a..ae35118 100644
--- a/src/Hedgehog/Hedgehog.fsproj
+++ b/src/Hedgehog/Hedgehog.fsproj
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>netstandard1.6;netstandard2.0;net45</TargetFrameworks>
+    <TargetFrameworks>netstandard1.6;netstandard2.0</TargetFrameworks>
     <Version>0.8.4</Version>
     <Description>Release with confidence.</Description>
     <Authors>Jacob Stanley;Nikos Baxevanis</Authors>
diff --git a/tests/Hedgehog.CSharp.Tests/Hedgehog.CSharp.Tests.csproj b/tests/Hedgehog.CSharp.Tests/Hedgehog.CSharp.Tests.csproj
index fb4c992..2b078b2 100644
--- a/tests/Hedgehog.CSharp.Tests/Hedgehog.CSharp.Tests.csproj
+++ b/tests/Hedgehog.CSharp.Tests/Hedgehog.CSharp.Tests.csproj
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>netcoreapp3.0;net461</TargetFrameworks>
+    <TargetFrameworks>netcoreapp3.0</TargetFrameworks>
     <ApplicationIcon />
     <OutputType>Library</OutputType>
     <StartupObject />
diff --git a/tests/Hedgehog.Tests/Hedgehog.Tests.fsproj b/tests/Hedgehog.Tests/Hedgehog.Tests.fsproj
index ac4a59c..a89ea3b 100644
--- a/tests/Hedgehog.Tests/Hedgehog.Tests.fsproj
+++ b/tests/Hedgehog.Tests/Hedgehog.Tests.fsproj
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>netcoreapp3.0;net461</TargetFrameworks>
+    <TargetFrameworks>netcoreapp3.0</TargetFrameworks>
     <OutputType>Library</OutputType>
   </PropertyGroup>
   <ItemGroup>

(I'm not going to propose that change above.) As long as our GitHub Action(s) run fine we should be fine.

moodmosaic commented 3 years ago

@TysonMN, another though... the net45 target we have perhaps should be net452 as per https://dotnet.microsoft.com/download/dotnet-framework.

According to that URL, .NET Framework 4.5 and .NET Framework 4.5.1 are no longer supported.

TysonMN commented 3 years ago

I think you are conflating two things.

I think "support" here means whether Microsoft will release patch updates to fix (security) bugs. Independent of that, you can create an application that includes any of those targets. I am not completely sure about this though, so maybe I am wrong here.

For 4.5.x, my system has installed

For 4.6.1, my system has installed

Since you are unable to target 4.5 and 4.6.1, my guess is that you don't have the corresponding (multi-)targeting packs installed.

moodmosaic commented 3 years ago

...Well, I think that Microsoft is trying to 'fix' all that with .NET 5 :wink: