search

microsoft / PowerToys

Windows system utilities to maximize productivity
MIT License
111.37k stars 6.56k forks source link

All PowerLauncher tests are not ran in CI #5879

Closed royvou closed 4 years ago

royvou commented 4 years ago

After i've checked #5878 (Which fixes the tests locally and add it to CI) i noticed in Azure Devops, all nunit (Launcher Plugins) are throwing an exception and thus are not ran (or showing an error).

ℹ Computer information

📝 Provide detailed reproduction steps (if any)

  1. Go to a build in Azure Devops https://dev.azure.com/ms/PowerToys/_build/results?buildId=100826&view=logs&j=0f660c0a-4423-5c5d-276b-f98b39c69e13&t=9445b2eb-6ed1-5959-53ea-8ccb71fe5423
  2. Look at the error logs

✔️ Expected result

❌ Actual result

crutkas commented 4 years ago 
The NUnit 3 driver encountered an error while executing reflected code.
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 83
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 293
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 110
InnerException: System.InvalidCastException: Unable to cast transparent proxy to type 'System.Web.UI.ICallbackEventHandler'.
   at NUnit.Framework.Api.FrameworkController.LoadTestsAction..ctor(FrameworkController controller, Object handler)
royvou commented 4 years ago 
Test run will use DLL(s) built for framework .NETFramework,Version=v4.7.2 and platform X64. Following DLL(s) do not match framework/platform settings.
Microsoft.Plugin.Program.UnitTests.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Microsoft.Plugin.Program.UnitTests.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Wox.Test.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Wox.Test.dll is built for Framework .NETCoreApp,Version=v3.1 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.

Looks like the test runner does not like mixing .NETFramework 4.7.2 and .NET Core 3.1

crutkas commented 4 years ago

That could be it. Why do these run local then is the larger q.

crutkas commented 4 years ago

@ryanbodrug-microsoft any ideas

royvou commented 4 years ago

FYI seems like somebody else is having this issue as well. https://github.com/microsoft/vstest/issues/2310 Their "Workaround" is to execute VSTest@2 twice once for all .NET framework and once for all .NET Core assemblies.

Probably Visual Studio is running vstest.console.exe once per project to prevent such conflicts.

Note: The tests are also testing dll's inside the /obj/ folder, seems incorrect as well (Even though they should be excluded)

ryanbodrug-microsoft commented 4 years ago

Ouch.. thanks @royvou. I'll create an issue and get these back running.

ryanbodrug-microsoft commented 4 years ago

Ouch.. thanks @royvou. I'll create an issue and get these back running.

@NVM I just realized I was commenting on this issue. I can take this unless you've started working on it @royvou

royvou commented 4 years ago

Go ahead @ryanbodrug-microsoft :)