JUnit output not compatible with the jenkins "xUnit" plugin

[VirtualTim]

Trying to use the output jnit xml file in Jenkins generates the following:

WARNING: The file '/UnitTests.xml' is an invalid file.
WARNING: At line 3 of file:/UnitTests.xml:cvc-pattern-valid: Value '0.9729' is not facet-valid with respect to pattern '(([0-9]{0,3},)*[0-9]{3}|[0-9]{0,3})*(\.[0-9]{0,3})?' for type 'SUREFIRE_TIME'.
WARNING: At line 3 of file:/UnitTests.xml:cvc-complex-type.4: Attribute 'errors' must appear on element 'testsuite'.
WARNING: The result file '/UnitTests.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

It looks like the Jenkins plugin (https://plugins.jenkins.io/xunit/) expects:

1. only 3 decimal places after the . in the time attribute
2. an errors attribute, even if errors=0 on the testsuite

[VirtualTim]

I've created a pretty quick xslt to "massage" the xml to a compatible format. It's obviously not a "good" fix, but hopefully works until/if this issue is addressed.

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <!-- Note: added to workaround https://github.com/a7ex/xcresultparser/issues/37 -->
    <xsl:output method="xml" encoding="utf-8" omit-xml-declaration="no" indent="yes"/>

    <!-- Identity template: copy everything as-is -->
    <xsl:template match="@*|node()">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>

    <!-- Limit "time" to 3dp -->
    <xsl:template match="@time">
        <xsl:attribute name="time">
            <xsl:value-of select="format-number(number(.), '#0.000')"/>
        </xsl:attribute>
    </xsl:template>

    <!-- Add the "errors" attribute to every <testsuite|testsuites> -->
    <!-- Just set to 0, don't know if XCTest differentiates errors/failures -->
    <xsl:template match="testsuite|testsuites">
        <xsl:copy>
            <xsl:attribute name="errors">0</xsl:attribute>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>

</xsl:stylesheet>

[a7ex]

Hi Tim, that shouldn't be too difficult to fix. I wonder however what lead me to the decision to format the number with 4 fractional digits: https://github.com/a7ex/xcresultparser/blob/3929950c8f3ec93b57b7ac5d2362fc0c7d78c85f/Sources/xcresultparser/JunitXML.swift#L50C9-L50C47 Maybe it was just like that in an example file I used to implement Unit format. As to why Jenkins chokes on 4 fractional digits...? Odd. The fix is easy, we need just a 3, where we now have a 4 in the file, which I linked above.

The other issue, "errors". What is that supposed to mean? Compile errors or test failures? Right now the xml reflects the test failures in the "failures" attribute. Is "errors" Jenkins way of saying "failures" or are test failures and errors two different things? Xcode is clear about that. Errors are compile errors. If there is a compile error, then there are no tests at all. I can easily add an attribute "errors" with the same content as "failures", if that is what you want.

[VirtualTim]

Thanks a7ex. So the documentation is a bit vague, but my understanding is that, as junit comes from Java, failures are when an assert fails, ie: an AssertionError is thrown. errors are then when the test fails in some other, unhandled way. Like maybe you have a test that compares two images. It would be a failure if the images did not match, but an error if the known good image is missing. But I don't think XCode has that concept? Probably the closest thing would be is a test crashes. So it's probably fine to just have errors="0".

[a7ex]

So I'll add an "errors" property to the "testsuites" and "testsuite" xml nodes with a hard coded value of 0 and then change the fractional digits to 3 and then we can look how it goes.

[a7ex]

The changes are now implemented in version 1.6.5

[VirtualTim]

Awesome. That's for the quick turnaround.