[BUG] Test Explorer Does Not Show Status for Data Driven Tests

[acgritt]

Describe the Issue

I have some xUnit tests that are data driven, both InlineData and MemberData. Those tests do get run but their status is not reflected in the Test Explorer. Also the count/parameters do not get detailed out.

Steps To Reproduce

1. Create a data driven xUnit Test
2. Run Unit Test in Test Explorer

Expected Behavior

The status and details should show up for the run tests. The .NET Core Test Explorer extension I used to use prior to the C# Dev Kit included this behavior.

Environment Information

Item	Value
CPUs	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 2592)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	31.71GB (3.73GB free)
Process Argv	. --crash-reporter-id c77e07f2-3afd-401a-8c5c-c503b5977532
Screen Reader	no
VM	0%

[timheuer]

Hi @acgritt thanks for the report -- I have an InlineData/Theory test that shows results...is the xUnit pattern you are doing similar? Example repo: https://github.com/timheuer/TestSolution (in the TestXUnit project)

[acgritt]

@timheuer Looking at all of the tests I have that are failing to show up properly, it seems it is rather inconsistent. The one thing they all have in common though is that they are all using [MemberData(...)] attributes. Most of them are marked as internal. In the following screenshot there is 3 sets of data for the test and the test shows as passed but only matches one set of data:

In this next one it shows as not having been run and there are 7 sets of test data.

The test data largely consists of enums and objects. In the two examples I have both are using the same sets of objects.

Another item I have noticed is that as the tests run I see that the Test Explorer reports as NNN/1550 tests passed but then upon completion it is 1090/1090 tests passed. I am not sure if it is due to the complexity of the tests/data or the number of tests.

[timheuer]

Good additional info for us to track down @acgritt -- thanks for the additional info. I know you likely can't share this project but if you have a smaller isolated repro showing the similar pattern, we'd be happy to work off the same pattern to debug. I added some very simple MemberData to my own sample solution and they show up. https://github.com/timheuer/TestSolution/blob/main/TestXUnit/MathTests.cs#L15-L49

The NNN/1550 passed is likely related as a calc issue on the test runs. /cc @peterwald

[acgritt]

@timheuer I was able to replicate it on my personal computer in a simple repo. At least it appears the same on my Mac. I have not had a chance to pull and retest on a Windows PC but I do not expect this to be different there.

https://github.com/acgritt/TestExplorerBug

[timheuer]

Great, thanks for this...I see this now with this sample (on mac and windows):

https://github.com/microsoft/vscode-dotnettools/assets/4821/8edff09d-4f72-4e9f-9b08-5e4cf6553c99

The tests run, succeed, then the 'state' gets reset to not run for some reason.

[github-actions[bot]]

This issue has been marked as stale after 14 days of inactivity.@[@@], could you please take a look?

[ocallesp]

@ocallesp need to reproduce and investigate

[ocallesp]

I was able to reproduce it.

[ocallesp]

Similar case