GD-103: Show less stack trace for failing test when using `MSTest` assertions

[DrMouseInc]

Is your feature request related to a problem? Please describe. As a future QOL enhancement, consider showing less stack if the user is using assertions from the MSTest.TestFramework nuget package. Currently, having these two methods (fiddling with the i is just to have a few more lines in there)

[TestCase]
public void Test1() {
    int i = 2;
    i += 2;

    Assertions.AssertInt(i).IsEqual(1);
}

[TestCase]
public void Test2() {
    int i = 2;
    i += 2;

    Assert.AreEqual(1, i);
}

Running them from the Visual Studio test explorer gives the gdUnit assertions a nice short stack trace with the actual line of the failing assertion easily found and clickable:

while the MSTest assertion has a few more lines and the user has to do more mental parsing to find the actual line that they should click to get straight to the failing assertion

As a reference, running the test through the normal MSTest runner also gives a nice clean message

Describe the solution you'd like I would like the gdUnit test runner to cut out the other clickable lines from the stack trace and leave only the one that goes straight to the assertion.

Describe alternatives you've considered Alternatives are the user moves to using the gdUnit assertions or continues to have to mentally parse a few more lines for every failed test.

Additional context None

[MikeSchulze]

If I understand you correctly, you have mixed GdUnit4 assertions and MSTest assertions?

The GdUnit4 test adapter only shows the stack trace generated by the assertion.

It is not intended to mix different assertion APIs. Is there a particular reason why you are using GdUnit4 and MSTest Assertion here?

What I found out is that the MSTest test adapter performs an additional stack trace trim. https://github.com/microsoft/testfx/blob/main/src/Adapter/MSTest.TestAdapter/Execution/ExceptionHelper.cs#L101

[DrMouseInc]

No, I only use MSTest assertions in my actual project. This was just to show the difference of the stack trace that the VS test explorer window shows for an equivalent assertion.

[MikeSchulze]

I don't get your point, you only use MSTest assertions but using the GdUnit4 test framework? Why, can you give me more insides why you are using this kind of mixed?

Can you show me your project dependencies please?

[DrMouseInc]

I was under the assumption that I need the GdUnit4 framework to run tests against Godot? I've adapted my current test setup from your example project. And I use MSTest assertions because I always use them, including professionally.

So I have both the actual Godot script code and the tests under one .csproj which has the following dependencies

<ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="gdUnit4.api" Version="4.2.3" />
    <PackageReference Include="gdUnit4.test.adapter" Version="1.1.1" />
    <PackageReference Include="MSTest.TestFramework" Version="3.3.1" />
 </ItemGroup>

This does skirt around the issue that I think there is a need for a better example project, showing multiple tests, how to use ISceneRunner properly, using a single project for everything or one for the Godot stuff and one for testing, etc. etc. But that's for another conversation.

[MikeSchulze]

I was under the assumption that I need the GdUnit4 framework to run tests against Godot?

Yes, the gdUnit4.api provides the test runner and assertions to work with Godot code.

And I use MSTest assertions because I always use them, including professionally.

I understand your point, and it should be no problem to use the MSTest assertions inside the GdUnit4 tests. Ultimately, the assertions throw an exception that the GdUnit4 test framework catches and reports.

In summary, it therefore makes sense to shorten the stack trace as it is done in MSTest adapter.

Thanks for suggesting these improvements.

[MikeSchulze]

@DrMouseInc you can upgrade to 4.2.4-rc3

[DrMouseInc]

@MikeSchulze Looks much better, thank you again!