VSTest doesn't support multiple versions of a test framework

[Evangelink]

Description

As part of MSTest v3 we did some breaking changes compared to v2. If a project contains a mix of v2 and v3 then some weird errors will happen at runtime.

Steps to reproduce

Create a solution with 1 project using MSTest v2 and 1 project using MSTest v3 then run dotnet test or vstest.console.exe.

Expected behavior

Tests are passing.

Actual behavior

Tests are failing

Diagnostic logs

========== Starting test run ==========
Multiple versions of same extension found. Selecting the highest version.
  Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter : 3.400.24.21801
Test Run deployment issue: The assembly or module 'Microsoft.Extensions.Configuration, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' was not found. Reason: Could not load file or assembly 'Microsoft.Extensions.Configuration, Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Test Run deployment issue: The assembly or module 'BuildXL.Processes, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6212d9137135ce5d' was not found. Reason: Could not load file or assembly 'BuildXL.Processes, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6212d9137135ce5d' or one of its dependencies. The system cannot find the file specified.
Test Run deployment issue: The assembly or module 'BuildXL.Utilities.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6212d9137135ce5d' was not found. Reason: Could not load file or assembly 'BuildXL.Utilities.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6212d9137135ce5d' or one of its dependencies. The system cannot find the file specified.
An exception occurred while invoking executor 'executor://mstestadapter/v2': Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
Stack trace:
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestDeployment.GetDeploymentInformation(String source)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTestsInSource(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, String source, Boolean isDeploymentDone) in /_/src/Adapter/MSTest.TestAdapter/Execution/TestExecutionManager.cs:line 246
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, Boolean isDeploymentDone) in /_/src/Adapter/MSTest.TestAdapter/Execution/TestExecutionManager.cs:line 146
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, TestRunCancellationToken runCancellationToken) in /_/src/Adapter/MSTest.TestAdapter/Execution/TestExecutionManager.cs:line 85
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.<>c__DisplayClass8_0.<RunTests>b__1(TestRunCancellationToken testRunToken) in /_/src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs:line 57
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.<>c__DisplayClass11_0.<RunTestsFromRightContext>g__DoRunTests|0() in /_/src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs:line 126
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.RunTestsFromRightContext(IFrameworkHandle frameworkHandle, Action`1 runTestsAction) in /_/src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs:line 134
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionDecorators.SerialTestRunDecorator.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle) in /_/src/Microsoft.TestPlatform.Common/ExtensionDecorators/SerialTestRunDecorator.cs:line 46
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithTests.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUri, RunContext runContext, IFrameworkHandle frameworkHandle) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Execution/RunTestsWithTests.cs:line 73
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.<>c__DisplayClass46_0.<RunTestInternalWithExecutors>b__0() in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Execution/BaseRunTests.cs:line 464
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.<>c__DisplayClass0_0.<Run>b__0() in /_/src/Microsoft.TestPlatform.PlatformAbstractions/net462/System/PlatformThread.cs:line 29
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.Run(Action action, PlatformApartmentState apartmentState, Boolean waitForCompletion) in /_/src/Microsoft.TestPlatform.PlatformAbstractions/net462/System/PlatformThread.cs:line 47
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.TryToRunInStaThread(Action action, Boolean waitForCompletion) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Execution/BaseRunTests.cs:line 680
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Execution/BaseRunTests.cs:line 464

========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 972 ms ==========

[nohwnd]

related? https://github.com/microsoft/vstest/issues/3475

[nohwnd]

Closing, will continue in the linked issue.