All PowerLauncher tests are not ran in CI

[royvou]

After i've checked #5878 (Which fixes the tests locally and add it to CI) i noticed in Azure Devops, all nunit (Launcher Plugins) are throwing an exception and thus are not ran (or showing an error).

ℹ Computer information

• PowerToys version: N/A
• PowerToy Utility: N/A
• Running PowerToys as Admin:
• Windows build number: [run "winver"]

📝 Provide detailed reproduction steps (if any)

1. Go to a build in Azure Devops https://dev.azure.com/ms/PowerToys/_build/results?buildId=100826&view=logs&j=0f660c0a-4423-5c5d-276b-f98b39c69e13&t=9445b2eb-6ed1-5959-53ea-8ccb71fe5423
2. Look at the error logs

✔️ Expected result

• No Excecptions

  Running all tests in d:\a\1\s\src\modules\launcher\Plugins\Microsoft.Plugin.Program.UnitTests\bin\x64\Release\netcoreapp3.1\Microsoft.Plugin.Program.UnitTests.dll
  Exception NUnit.Engine.NUnitEngineException, Exception thrown executing test

• Succesfull tests

❌ Actual result

• Exceptions.

[crutkas]

The NUnit 3 driver encountered an error while executing reflected code.
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 83
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 293
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 110
InnerException: System.InvalidCastException: Unable to cast transparent proxy to type 'System.Web.UI.ICallbackEventHandler'.
   at NUnit.Framework.Api.FrameworkController.LoadTestsAction..ctor(FrameworkController controller, Object handler)

[royvou]

Test run will use DLL(s) built for framework .NETFramework,Version=v4.7.2 and platform X64. Following DLL(s) do not match framework/platform settings.
Microsoft.Plugin.Program.UnitTests.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Microsoft.Plugin.Program.UnitTests.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Wox.Test.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Wox.Test.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.

Looks like the test runner does not like mixing .NETFramework 4.7.2 and .NET Core 3.1

[crutkas]

That could be it. Why do these run local then is the larger q.

[crutkas]

@ryanbodrug-microsoft any ideas

[royvou]

FYI seems like somebody else is having this issue as well. https://github.com/microsoft/vstest/issues/2310 Their "Workaround" is to execute VSTest@2 twice once for all .NET framework and once for all .NET Core assemblies.

Probably Visual Studio is running vstest.console.exe once per project to prevent such conflicts.

Note: The tests are also testing dll's inside the /obj/ folder, seems incorrect as well (Even though they should be excluded)

[ryanbodrug-microsoft]

Ouch.. thanks @royvou. I'll create an issue and get these back running.

[ryanbodrug-microsoft]

Ouch.. thanks @royvou. I'll create an issue and get these back running.

@NVM I just realized I was commenting on this issue. I can take this unless you've started working on it @royvou

[royvou]

Go ahead @ryanbodrug-microsoft :)