Failure message in the XML report is incomplete from Go 1.14 up

[generalmimon]

As of Go 1.14, the go test -v command started putting the --- FAIL: line below the message (not above as before). Compare the go1.13.15 and go1.14 console output:

pp@DESKTOP-MIPASSQ MINGW64 /c/temp/go-test
$ go1.13.15 version
go version go1.13.15 windows/amd64

pp@DESKTOP-MIPASSQ MINGW64 /c/temp/go-test
$ go1.14 version
go version go1.14 windows/amd64

pp@DESKTOP-MIPASSQ MINGW64 /c/temp/go-test
$ go1.13.15 test -v
=== RUN   TestExample
--- FAIL: TestExample (0.00s)
    example_test.go:9:
                Error Trace:    example_test.go:9
                Error:          Not equal:
                                expected: 1
                                actual  : 2
                Test:           TestExample
FAIL
exit status 1
FAIL    _/C_/temp/go-test       0.029s

pp@DESKTOP-MIPASSQ MINGW64 /c/temp/go-test
$ go1.14 test -v
=== RUN   TestExample
    TestExample: example_test.go:9:
                Error Trace:    example_test.go:9
                Error:          Not equal:
                                expected: 1
                                actual  : 2
                Test:           TestExample
--- FAIL: TestExample (0.00s)
FAIL
exit status 1
FAIL    _/C_/temp/go-test       0.033s

Unified diff:

- $ go1.13.15 test -v
+ $ go1.14 test -v
  === RUN   TestExample
- --- FAIL: TestExample (0.00s)
-     example_test.go:9:
+     TestExample: example_test.go:9:
                  Error Trace:    example_test.go:9
                  Error:          Not equal:
                                  expected: 1
                                  actual  : 2
                  Test:           TestExample
+ --- FAIL: TestExample (0.00s)
  FAIL
  exit status 1
- FAIL    _/C_/temp/go-test       0.029s
+ FAIL    _/C_/temp/go-test       0.033s

The above example is simulated by this simple code (you need to run go get github.com/stretchr/testify/assert before):

example_test.go

package example

import (
        "testing"
        "github.com/stretchr/testify/assert"
)

func TestExample(t *testing.T) {
        assert.EqualValues(t, 1, 2)
}

By the incomplete failure message, I mean this (diffing go1.13.15 test -v | go-junit-report and go1.14 test -v | go-junit-report using the console outputs you've seen before):

 <?xml version="1.0" encoding="UTF-8"?>
 <testsuites>
-   <testsuite tests="1" failures="1" time="0.029" name="_/C_/temp/go-test">
+   <testsuite tests="1" failures="1" time="0.033" name="_/C_/temp/go-test">
        <properties>
            <property name="go.version" value="go1.15.7"></property>
        </properties>
        <testcase classname="go-test" name="TestExample" time="0.000">
-           <failure message="Failed" type="">example_test.go:9: &#xA;Error Trace:&#x9;example_test.go:9&#xA;Error:      &#x9;Not equal: &#xA;            &#x9;expected: 1&#xA;            &#x9;actual  : 2&#xA;Test:       &#x9;TestExample</failure>
+           <failure message="Failed" type="">    TestExample: example_test.go:9: </failure>
        </testcase>
    </testsuite>
 </testsuites>

So go-junit-report apparently just takes the first line of the message from go1.14 test -v output and the rest is ignored. (Also notice that the go1.14 prepends TestExample: to the message - this should be best omitted from the XML output because it merely duplicates the <testcase>.name, and it wasn't there in go1.13 or older.)

I think it's quite important to get this fixed if you plan to maintain this project, because it seems as all subsequent Go versions will behave like go1.14: I've just checked the latest available version go1.16.2 and it emits the exact same format as go1.14.

[smailz]

agree!

[liyaka]

Any workaround for the meantime?... Or planned fix?

[jstemmer]

Thanks for the detailed report. This looks indeed like a bug with the output parsing. I suspect it may be caused by the mix of spaces and tabs in the output.

I couldn't reproduce the part where the test name is prepended to the first line of output. After a bit of digging it looks like this was introduced in go1.14 but removed again in go1.14.6.

[janisz]

What's the status of this?

[jstemmer]

This has been fixed in the v2 branch.

[janisz]

How can I use v2? It's not go gettable :(

[jstemmer]

I haven't cut a release yet from the v2 branch, but there should be a v2 beta soon. If you can't wait, you'll have to build it manually.

[jstemmer]

This issue has been fixed in the recently released v2.0.0-beta.