Extra testcases in Junit xml output with pytest

[Hebbeman]

When running flaky together with pytest using junit xml output:

pytest --junit-xml=result.xml

it appears like each rerun of a flaky test is registered as an extra testcase in the junit xml output with no attributes except time, for example: <testcase time="0.0019998550415"></testcase>

Is this working as intended?

[comandrei]

I've had this happen when using pytest==2.9.1 When upgrading to 3.0.3, this is no longer an issue.

[Hebbeman]

I upgraded to pytest 3.0.3 (was running 3.0.2), but problem persist :(

With a simple hack I can remove the empty testcases before I publish the test result. However, it would be neater if flaky could handle this. Or perhaps I need to upgrade some other dependency?

[kalidasya]

@Hebbeman can you share your simple hack? I agree flaky should not spam the report with empty test case elements.

[Hebbeman]

@kalidasya The "hack" simply modifies the JUnitXML file

import shutil
import lxml.etree as et

# The integration of pytest plugin junitxml and python module 'flaky'
# give rise to 'extra' testcases in the result junit-xml file.
#
# This function is a workaround to remove these 'extras'
def delete_flaky_reruns(junitxml_file):
    doc = et.parse(junitxml_file)
    suite = doc.getroot()
    for case in suite:
        try:
            # If the case lack name it is added because of flaky rerun
            name = case.attrib['name']
        except KeyError:
            print "Removing faulty testcase"
            case.getparent().remove(case)

    doc.write(junitxml_file)

[kalidasya]

@Hebbeman thanks, I thought you hooked it with pytest and you do this on that report object, but indeed we can do it afterwards as well. Thanks!