merge JUnit XML with TestSuites vs TestSuite as root element should not result in nested.

[ZSmallX]

Description: I got the following nested results when merge one JUnit XML with root element testsuite and another JUnit XML with root element testsuites.

JUnit XML with testsuite(name test_suite.xml in code):

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="com.example.unittestdemo.ExampleUnitTest" tests="1" skipped="0" failures="0" errors="0" timestamp="2020-11-19T14:34:55" hostname="MyHost" time="0.002">
  <properties/>
  <testcase name="division_isCorrect" classname="com.example.unittestdemo.ExampleUnitTest" time="0.002"/>
   <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

JUnit XML with testsuites(name test_suites.xml in code):

<testsuites tests="1" skipped="0" failures="0" errors="0" timestamp="2020-12-03T07:55:23" hostname="MyHost"
            time="0.005">
    <testsuite name="com.example.base.BaseUnitTest" tests="1" skipped="0" failures="0" errors="0"
               timestamp="2020-12-03T07:55:23" hostname="MyHost" time="0.005">
        <properties/>
        <testcase name="division_isCorrect" classname="com.example.base.BaseUnitTest" time="0.005"/>
        </testcase>
        <system-out><![CDATA[]]></system-out>
        <system-err><![CDATA[]]></system-err>
    </testsuite>
</testsuites>

And I run the code:

xml_suite_path = "test_suite.xml"
xml_suites_path = "test_suites.xml"

xml_suite = JUnitXml.fromfile(filepath=xml_suite_path)
xml_suites = JUnitXml.fromfile(filepath=xml_suites_path)

merged_xml = xml_suite + xml_suites

merged_xml.write("merged.xml")

Actual:

<?xml version='1.0' encoding='utf-8'?>
<testsuites>
    <testsuite name="com.example.unittestdemo.ExampleUnitTest" tests="1" skipped="0" failures="0" errors="0"
               timestamp="2020-11-19T14:34:55" hostname="MyHost" time="0.002">
        <properties/>
        <testcase name="division_isCorrect" classname="com.example.unittestdemo.ExampleUnitTest" time="0.002"/>
        <system-out><![CDATA[]]></system-out>
        <system-err><![CDATA[]]></system-err>
    </testsuite>
    <testsuites tests="1" skipped="0" failures="0" errors="0" timestamp="2020-12-03T07:55:23" hostname="MyHost"
                time="0.005">
        <testsuite name="com.example.base.BaseUnitTest" tests="1" skipped="0" failures="0" errors="0"
                   timestamp="2020-12-03T07:55:23" hostname=MyHost" time="0.005">
            <properties/>
            <testcase name="division_isCorrect" classname="com.example.base.BaseUnitTest" time="0.005"/>
            <system-out><![CDATA[]]></system-out>
            <system-err><![CDATA[]]></system-err>
        </testsuite>
    </testsuites>
</testsuites>

Expected:

<testsuites tests="5" failures="1" errors="0" skipped="1" time="0.004">
    <testsuite name="com.example.unittestdemo.ExampleUnitTest" tests="1" skipped="0" failures="0" errors="0"
               timestamp="2020-11-19T14:34:55" hostname="MyHost" time="0.002">
        <properties/>
        <testcase name="division_isCorrect" classname="com.example.unittestdemo.ExampleUnitTest" time="0.002"/>
        <system-out><![CDATA[]]></system-out>
        <system-err><![CDATA[]]></system-err>
    </testsuite>

    <testsuite name="com.example.base.BaseUnitTest" tests="1" skipped="0" failures="0" errors="0"
               timestamp="2020-12-03T07:55:23" hostname=MyHost" time="0.005">
        <properties/>
        <testcase name="division_isCorrect" classname="com.example.base.BaseUnitTest" time="0.005"/>
        <system-out><![CDATA[]]></system-out>
        <system-err><![CDATA[]]></system-err>
    </testsuite>
</testsuites>

My Solution: And currently I solved this problem by converting all testsuite JUnit XML to testsuites JUnit XML and then update the statistics.

def convert_to_test_suites_if_needed(xml):
    if isinstance(xml, TestSuite):
        new_xml = JUnitXml()
        new_xml.add_testsuite(xml)
        return new_xml

    return xml

xml_suite_path = "test_suite.xml"
xml_suites_path = "test_suites.xml"

xml_suite = convert_to_test_suites_if_needed(JUnitXml.fromfile(filepath=xml_suite_path))
xml_suites = convert_to_test_suites_if_neededJUnitXml.fromfile(filepath=xml_suites_path))

merged_xml = xml_suite + xml_suites

merged_xml.update_statistics()

merged_xml.write("merged.xml")

I think this could be done in junitparser side to make it compatibile with merging testsuite and testsuites and get the right statistics.

[dyollb]

Here is some example data which shows the issue:

If you merge first the mypy xml, then the pytest xml, the result is wrong (nested testsuites). If you switch the order (pytest with two testsuites, then mypy) the result is as expected.

The workaround proposed above behaves correctly for both cases.

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="mypy" skips="0" tests="1" time="1.009">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_8-win32" time="1.009">
  </testcase>
</testsuite>

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite errors="0" failures="0" hostname="GAIA02" name="pytest" skipped="0" tests="2" time="0.630" timestamp="2023-08-18T14:49:23.787800">
        <testcase classname="tests.test_crop_box" file="tests\test_crop_box.py" line="9" name="test_crop_box" time="0.094"></testcase>
        <testcase classname="tests.test_utils" file="tests\test_utils.py" line="5" name="test_collect_files" time="0.001"></testcase>
    </testsuite>
</testsuites>