spekt / xunit.testlogger

XUnit logger for vstest platform
MIT License
73 stars 15 forks source link

Nested classes cause UnknownNamespace.UnknownType to be outputted #48

Closed pageyboy closed 8 months ago

pageyboy commented 8 months ago

Hi,

Really like the project and it has really helped with outputting test results into a legible format for our build. However, I have run into what I think is a defect. I have taken to organizing large Test Class files into nested classes (separated by method/property etc.). After doing this and running xunit.testlogger the number of tests is misreported and the type is not reported correctly. Happy to help out with debugging or fixing but I've struggled to get the tests to run in the solution - any advice you have here in getting your dev environment setup would be appreciated.

See below for minimum reproducible issue: Test Class 1

using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using src;

namespace srcTests
{
    [TestClass]
    public class SourceClassTests
    {
        [TestMethod]
        public void Should_ReturnTwo()
        {
            var sut = new SourceClass();

            sut.Result.Should().Be(2);
        }
    }
}

Test Class 2 (separated to allow for easier navigation in VS)

using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using src;

namespace srcTests
{
    [TestClass]
    public class SourceClassTests2
    {
        [TestClass]
        public class Constructor : SourceClassTests2
        {
            [TestMethod]
            public void Should_DoSomething_When_SomeState()
            {
                var sut = new SourceClass();
                sut.Result.Should().Be(2);
            }
        }

        [TestMethod]
        public void Should_ReturnTwo()
        {
            var sut = new SourceClass();

            sut.Result.Should().Be(2);
        }
    }
}

These 3 tests should produce 3 test results but instead I see 4 in total and also the type as Test collection for UnknownNamespace.UnknownType for the nested classes:

<assemblies timestamp="09/04/2023 14:14:34">
  <assembly name="U:\_Source Code\Tests\UnknownNamespace\srcTests\bin\Debug\srcTests.dll" run-date="2023-09-04" run-time="14:14:34" config-file="U:\_Source Code\Tests\UnknownNamespace\srcTests\bin\Debug\srcTests.dll.config" total="4" passed="4" failed="0" skipped="0" time="0.968" errors="0">
    <errors />
    <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for srcTests.SourceClassTests" time="0.089">
      <test name="Should_ReturnTwo" type="srcTests.SourceClassTests" method="Should_ReturnTwo" time="0.0890944" result="Pass">
        <traits />
      </test>
    </collection>
    <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for srcTests.SourceClassTests2" time="0.000">
      <test name="Should_ReturnTwo" type="srcTests.SourceClassTests2" method="Should_ReturnTwo" time="0.0001116" result="Pass">
        <traits />
      </test>
    </collection>
    <collection total="2" passed="2" failed="0" skipped="0" name="Test collection for UnknownNamespace.UnknownType" time="0.000">
      <test name="Should_DoSomething_When_SomeState" type="UnknownNamespace.UnknownType" method="Should_DoSomething_When_SomeState" time="0.0000943" result="Pass">
        <traits />
      </test>
      <test name="Should_ReturnTwo" type="UnknownNamespace.UnknownType" method="Should_ReturnTwo" time="0.0000035" result="Pass">
        <traits />
      </test>
    </collection>
  </assembly>
</assemblies>
pageyboy commented 8 months ago

PR for UnknownNamespace fix added here: https://github.com/spekt/testlogger/pull/41.

The issue mentioned about running the test multiple times is expected due to the inheritance and so this is not an issue with testlogger but more my design.

Additionally, I was able to build the solution in the end but I needed to add quotes around the test asset file paths. Not sure why this seems to be required for my environment:

  <PropertyGroup>
    <TestRestoreConfig>"$(MSBuildThisFileDirectory)../assets/NuGet.$(Configuration).config"</TestRestoreConfig>
    <TestFullProject>"$(MSBuildThisFileDirectory)../assets/Xunit.Xml.TestLogger.NetFull.Tests/Xunit.Xml.TestLogger.NetFull.Tests.csproj"</TestFullProject>
    <TestCoreProject>"$(MSBuildThisFileDirectory)../assets/Xunit.Xml.TestLogger.NetCore.Tests/Xunit.Xml.TestLogger.NetCore.Tests.csproj"</TestCoreProject>
  </PropertyGroup>
codito commented 8 months ago

@pageyboy thanks a lot for the PR!

I was able to build the solution in the end but I needed to add quotes around the test asset file paths

This shows up if the repo is cloned in a directory with whitespace in path. E.g., c:\my repos\xunit.testlogger. Fixed with https://github.com/spekt/xunit.testlogger/pull/49

codito commented 8 months ago

@pageyboy fix is available in nuget: https://www.nuget.org/packages/XunitXml.TestLogger/3.1.17. Please try it.