microsoft / testfx

MSTest framework and adapter
MIT License
714 stars 253 forks source link

Repo has unit test runtime dependency on deprecated .NET Core version #642

Closed jimmylewis closed 2 years ago

jimmylewis commented 4 years ago

Description

When cloning and running tests in this repo, there is a runtime dependency on having .NET Core 1.1 installed (specifically, for ValidateTimeoutTestsNetCore). If .NET Core 1.1 is not installed, this test will fail unintuitively - it will say that 0 of the test cases timed out (expected 2). In reality, they never ran (or failed to start).

.NET Core 1.1 is beyond its support lifecycle, so it will be increasingly common for developers to not have it present.

Steps to reproduce

What steps can reproduce the defect?

On a clean machine, install VS2019 (it no longer offers any way to acquire .NET Core 1.1), clone this repo, build, and run tests.

Expected behavior

Tests should all pass, or should fail with an accurate message.

Actual behavior

Failed ValidateTimeoutTestsNetCore Error Message: Test method MSTestAdapter.Smoke.E2ETests.TimeoutTests.ValidateTimeoutTestsNetCore threw exception: Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<2>. Actual:<0>. Stack Trace: at Microsoft.MSTestV2.CLIAutomation.CLITestBase.ValidateFailedTestsCount(Int32 expectedFailedTestsCount) in D:\a\1\s\test\E2ETests\Automation.CLI\CLITestBase.cs:line 152 at MSTestAdapter.Smoke.E2ETests.TimeoutTests.Validate(String testAssembly, String fileToValidate) in D:\a\1\s\test\E2ETests\Smoke.E2E.Tests\TimeoutTests.cs:line 38 at MSTestAdapter.Smoke.E2ETests.TimeoutTests.ValidateTimeoutTestsNetCore() in D:\a\1\s\test\E2ETests\Smoke.E2E.Tests\TimeoutTests.cs:line 29

Workaround

Find and install the .NET Core 1.1 runtime.

mayankbansal018 commented 4 years ago

We understand that this is an issue for machines with VS 2019, & no .Net Core 1.1 available. Unfortunately we are not able to pick this up at this time. I've marked it as up for grabs, & will be happy to take up contributions from community.

eriawan commented 4 years ago

@mayankbansal018

I would like to pick this up to prepare for VS 2019.

I also will update the Windows 10 SDK support to at least supporting 1903 build 18362, because current master branch code has support for Windows 10 14393. According to Windows 10 lifecycle, this Windoows 10 build of 14393 is also not supported anymore.

Evangelink commented 2 years ago

I believe this is now fixed, can you confirm @Haplois ?

Haplois commented 2 years ago

Yes :)