search

xunit / visualstudio.xunit

VSTest runner for xUnit.net (for Visual Studio Test Explorer and dotnet test)
https://xunit.net/
Other
148 stars 80 forks source link

No test found in a WinUI 3 Unit Test App #423

Open JulienTheron opened 1 month ago

JulienTheron commented 1 month ago

Hi there,

We're currently porting a UWP app to WinUI 3, and we wanted to port the xUnit tests as well. Creating a standard xUnit test project can discover and run the tests, but some of them fails because we're using APIs that that require package identity. Since there is no template for creating a xUnit-specific WinUI 3 Test App, I used the MSTest one and edited the referenced packages as follows:

<ItemGroup>
  <PackageReference Include="Microsoft.WindowsAppSDK" Version="1.6.240923002" />
  <PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.11.1">
    <ExcludeAssets>build</ExcludeAssets>
  </PackageReference>
  <PackageReference Include="xunit" Version="2.9.2" />
  <PackageReference Include="xunit.runner.visualstudio" Version="2.8.2">
    <PrivateAssets>all</PrivateAssets>
    <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
  </PackageReference>
  <Manifest Include="$(ApplicationManifest)" />
</ItemGroup>

The projects builds without errors, and I can see the tests in Visual Studio's Test Explorer, but they do not run as they're not discovered at runtime: Test discovery finished: 0 Tests found in 1,2 sec

Is this a known issue?

Thanks in advance.

bradwilson commented 1 month ago

I don't have any experience with WinUI 3.

Have you done the list of steps illustrated here? https://learn.microsoft.com/windows/apps/winui/winui3/testing/#how-do-i-test-non-winui-functionality-in-my-app

PeterDraex commented 1 month ago

I'm was having the same issue in a ASP.NET MVC 8 project. The test is visible in Test Explorer, but they don't run when I start them from Visual Studio interface. Running through dotnet test working ok.

I've tried all the solutions I could find on the internet, no luck. My <PackageReference /> section in the project file looked like this:

<!-- BAD version -->
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="8.0.10" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageReference Include="Microsoft.TestPlatform" Version="17.11.1" />
<PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.11.1" />
<PackageReference Include="MSTest" Version="3.6.1" />
<PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" Version="17.12.6" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

When I replaced that <PackageReference> section with this, tests started running ok:

<!-- GOOD version -->
<PackageReference Include="coverlet.collector" Version="6.0.2">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

I'm not sure which package specifically fixed the issue. @bradwilson, if you know, it would be great to add some error message to the console when this issue happens, or at least some mention in the README section on GitHub. Would have saved me couple hours. Thanks!

bradwilson commented 1 month ago

I don't even know what's wrong yet. I haven't seen a repro project, only partial .csproj files. 😉

JulienTheron commented 1 month ago

Here is a repro project based on the Unit Test App (WinUI 3 in Desktop) project template: xUnit WinUI 3.zip

Here's what I noticed:

  1. xUnit tests do not run inside Visual Studio.
  2. xUnit tests do run with dotnet test (ex: dotnet test TestProject.csproj -p:Platform=x64), but they're running as an unpackaged app, so APIs that need package identity fail.

Thanks for your help!

bradwilson commented 1 month ago

@JulienTheron I'm sorry, I don't know anything about how WinUI is supposed to work with VSTest. You're going to have to get the VSTest team and/or the WinUI team and/or the Test Explorer involved here. That fact that things work with dotnet test seems to imply the problem is related to Test Explorer.

It's also possible that WinUI just doesn't work with xUnit.net. It's not a supported scenario. I will work to fix the issue once one the problem has been identified (assuming it's possible), but I am personally incapable of identifying what the problem is. Someone else is going to need to figure that out, I'm afraid.

bradwilson commented 1 month ago

@PeterDraex I am not able to reproduce your problem, so please open a new issue for your specific problem (with a repro project).

In the meantime, here is what I got from basically just creating and empty MVC project + an xUnit.net project, testing with Microsoft.AspNetCore.Mvc.Testing. It works for me. vs423-aspnet.zip

I will note that your original package list includes several things that could cause failures, starting with the fact that you appear to be trying to mix MSTest and xUnit.net into the same project.