xunit / visualstudio.xunit

VSTest runner for xUnit.net (for Visual Studio Test Explorer and dotnet test)
https://xunit.net/
Other
144 stars 81 forks source link

Which is the latest supported version for VS 2019 #400

Closed tinmanjk closed 7 months ago

tinmanjk commented 7 months ago

According to a comment in issue #360 ( For what it's worth, we don't support VS 2019 any more as of 2.5.0. You'd probably need to stick to 2.4.x anyway, even if you upgraded to .NET 6.)

However, the Nuget package for 2.50 states: This project contains the Visual Studio runner for xUnit.net. It supports the built-in Test Explorer feature in Visual Studio 2019 16.10 and later. It supports .NET 6+ and .NET Framework 4.6.2+. It can run tests from xUnit.net 1.9.2 and later.

FWIW, 2.50 works fine on VS 2019. What is expected to break on 2.5x+ on VS 2019 or is it just not actively tested against?

bradwilson commented 7 months ago

It's not actively tested against or officially supported.


That said...

We're linked against NuGet package Microsoft.TestPlatform.ObjectModel version 17.8.0 (which is the latest release version as of this writing), which includes three DLLs: Microsoft.TestPlatform.CoreUtilities.dll, Microsoft.TestPlatform.PlatformAbstractions, and Microsoft.VisualStudio.TestPlatform.ObjectModel.dll. Looking at these DLLs in ILSpy, they are all marked as assembly version 15.0.0.0, despite having "informational" versions that indicate that they are 17.8.0.

My belief, though I haven't validated this, is that the interfaces provided here (in particular in the namespace Microsoft.VisualStudio.TestPlatform.ObjectModel and below, which is what we use) have been stable since Visual Studio 2017, hence the version 15 moniker. They expect that any version anybody has linked to since VS2017 RTM should be successfully silently upgradeable to anything later. In theory, this means that we are probably still usable within any version of VS2017 and VS2019, if we are in fact not taking advantage of anything that may have been added since then. And I have no reason to think we are, because the interfaces and types that we use haven't really changed in many years.

So the upshot is that it's very likely that we still work without issue in VS2019. We just don't test it, and if it's broken, we (probably) won't fix it. I'll say probably rather than being absolute, because if an issue is found and the fix is trivial and that fix doesn't impact anything else, I'd absolutely consider taking a PR to fix it. Since VS2019 16.11 is still in mainstream support, that would be a reasonable thing to consider.

Anything older than that is no longer in mainstream support, so anything affecting earlier versions of VS2019 (or any prior major versions, like VS2017) would definitely not be something we'd fix. Perhaps just as important that mainstream support for VS2019 ends in two months on April 9, 2024 which makes most of what I wrote here theoretical. In two months, I very likely would not accept any PRs to fix any issues w/ VS2019.

FWIW, given the existing mainstream support for VS2019, I recently went back to officially supporting VS2019 16.11 with the Roslyn Analyzers. That wasn't exactly trivial, but it wasn't that difficult either. (And unlike this scenario, it does actually require active work on my part to make that work, because the Roslyn APIs do evolve regularly. Once April arrives, I'm very likely to end up removing the VS2019 support I just put in at some point, because there is an active development cost associated with building for, and testing against, VS2019 with the analyzers.)

However, the Nuget package for 2.50 states: This project contains the Visual Studio runner for xUnit.net. It supports the built-in Test Explorer feature in Visual Studio 2019

This is because I forgot to update the README and/or NUSPEC files in a proper manner. This was something I fixed for 2.5.1. Sorry about the confusion.

image

bradwilson commented 7 months ago

Just to satisfy my suspicions, I rolled back to 15.0.0 of Microsoft.TestPlatform.ObjectModel and, with the exception of one compiler warning (a nullability change they made on a dictionary value), everything worked as expected running against VSTest 17.8.0:

image

I consider that to be verification that we're very likely to be fine in VS2017 and VS2019 and we're not taking advantage of anything that's new, so I'm going to close this.