Closed rouke-broersma closed 5 years ago
See also integration test for VsTestConsoleWrapper which has test logic to say that the dotnet core vstest runner is not supported by VsTestConsoleWrapper: https://github.com/Microsoft/vstest/blob/master/test/Microsoft.TestPlatform.AcceptanceTests/TranslationLayerTests/DiscoverTests.cs#L21 Which actually uses the TestUtilities IntegrationTestBase to test that the dotnet core runner is not supported. The integration testbase that includes logic to start up the dotnet core vstest runner..
@Mobrockers since you already have a POC, is it possible for you to pick this issue, we will be happy to help you.
@mayankbansal018 I can probably pick this up.
Description
We are building a mutation testing framework for dotnet framework and dotnet core. We are trying to integrate vstest using the Microsoft.TestPlatform.TranslationLayer and Microsoft.TestPlatform.Portable nuget packages. This works on windows by providing the vstest.console.exe path from the nuget package to VsTestConsoleWrapper which provides the path to the VsTestConsoleProcessManager, however this does not seem to work for vstest.console.dll because the wrapper does not detect that the dll should be run using using dotnet with vstest.console.dll as the argument instead of trying to start a process for the dll directly. The IntegrationTestBase in the Microsoft.TestPlatform.TestUtilities project contains logic to differentiate between starting a vstest.console.exe process and starting a dotnet vstest.console.dll process. We need this logic in the TranslationLayer to be able to add vstest integration in our mutation test framework.
Steps to reproduce
I also have a proof of concept of our vstest integration on this branch here: https://github.com/stryker-mutator/stryker-net/tree/183-vstest-integration/src/Stryker.Core/Stryker.Core/TestRunners/vstest
Expected behavior
vstest.console.dll starts up, discovers tests in test project and returns discovered tests to TranslationLayer. Logging is available to see what vstest.console.dll executed.
Actual behavior
vstest.console.dll tries to start up, immediately fails because OS does not know how to run a dll, no logging is available because vstest.console.dll never actually started
Diagnostic logs
No logs are available because vstest.console.dll never actually started.
Environment
OS: Windows 10 Enterprise Packages and versions: