catchorg / Catch2

A modern, C++-native, test framework for unit-tests, TDD and BDD - using C++14, C++17 and later (C++11 support is in v2.x branch, and C++03 on the Catch1.x branch)
https://discord.gg/4CWS9zD
Boost Software License 1.0
18.65k stars 3.05k forks source link

junit report sections mixed #2252

Open APN-Pucky opened 3 years ago

APN-Pucky commented 3 years ago

Description I have a test structure like this with several stacked sections:

TEST_CASE( "Testing works", "[TEST]" ) {
    SECTION("A") {
        SECTION("B1") {
            SECTION("C1") {
                REQUIRE(1==1);
            }
            SECTION("C2") {
                FAIL("fail");
            }
        }
        SECTION("B2")
        {
                FAIL("fail");
        }
    }
}

then $ test -r junit gives:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test_test" errors="0" failures="2" tests="3" hostname="tbd" time="9.3e-05" timestamp="2021-06-25T09:30:07Z">
    <testcase classname="test_test.global" name="Testing works/C2/B1/C1" time="0.000012" status="run"/>
    <testcase classname="test_test.global" name="Testing works/C2/B1/A" time="0.000026" status="run">
      <failure type="FAIL">
FAILED:
fail
at .../test.test.cc:15
      </failure>
    </testcase>
    <testcase classname="test_test.global" name="Testing works/B2/A" time="0.000006" status="run">
      <failure type="FAIL">
FAILED:
fail
at .../test.test.cc:20
      </failure>
    </testcase>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>

The name of the first test case now contains both sections C1 and C2.

Expected behavior I'd expect the ordering to be TEST_CASE/A/B/C/... and with out mixing.

Platform information:

Additional context The command -r xml orders the sections correctly.

LinuxDevon commented 3 years ago

Similar request to #886