Test explorer gets stuck at 'in progress' using xUnit `Theory` tests

[marklam]

Describe the bug

Test explorer gets stuck at 'in progress' using xUnit Theory tests. It seems to be specific to the Theory type test, where the test is run multiple times with different parameters.

The output window shows that the dotnet test run completed, but the tree still shows it as running.

The equivalent code in C# run via the C# dev kit doesn't get stuck.

Steps to reproduce

(sample repo linked below)

Run an xUnit Theory test from the VSCode UI. The "Test Explorer" pane and "Test Results" tree both show in progress, although the "Test Results" window contains:

WARNING: Overwriting results file: c:\Users\markl\AppData\Roaming\Code\User\workspaceStorage\947a947d1479406114d01bb6220c0a98\Ionide.Ionide-fsharp\TestResults\TestListProblemFs\TestListProblemFs.trx

Results File: c:\Users\markl\AppData\Roaming\Code\User\workspaceStorage\947a947d1479406114d01bb6220c0a98\Ionide.Ionide-fsharp\TestResults\TestListProblemFs\TestListProblemFs.trx

Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 3 ms - TestListProblemFs.dll (net6.0)

An equivalent C# project shows completion in the same windows.

Link to sample reproduction

https://github.com/marklam/TestListProblem

Expected behaviour

The tests should be marked as complete and passed

Screenshots

Machine info

• OS: Windows
• .NET SDK version: 6.0.414, 7.0.401
• Ionide version: v7.12.3

[TheAngryByrd]

@farlee2121

[farlee2121]

Interesting. The issue is that xUnit doesn't include the theory data in the test method name in the test results file. This means that all the theory tests try to register under the same name, and the explorer errors.

<!-- UnitTest.name includes the arguments -->
<UnitTest name="XUnitTests.theoryTest(x: 2, y: 3, sum: 4)" storage="c:\workspaces\dotnet\ionide-test-explorer-regressiontest\tests\xunittests\bin\debug\net6.0\xunittests.dll" id="bb7320e1-a388-2f06-e5b7-d89537c310e0">
  <Execution id="9100e2e7-a7d2-4b79-bc2f-692852593c7b" />
  <!-- TestMethod.name is method name only -->
  <TestMethod className="XUnitTests" name="theoryTest" codeBase="c:\workspaces\dotnet\Ionide-Test-Explorer-RegressionTest\tests\XUnitTests\bin\Debug\net6.0\XUnitTests.dll" adapterTypeName="executor://xunit/VsTestRunner2/netcoreapp"  />
</UnitTest>

I'll have to think through the consequences of using UnitTest.name instead of TestMethod.name and TestMethod.className. I think it'll be ok, but I'm not sure how the different libraries treat that field.

[farlee2121]

I figured out how to at least get parameterized tests running across frameworks, but some test filter quirks still make the experience pretty gnarly

1936

[TheAngryByrd]

@farlee2121 should we close this one?

[farlee2121]

I think so. It's now fully functional. I'll probably still take a shot at the nesting, but I think that's a separate scope.