search

jstemmer / go-junit-report

Convert Go test output to JUnit XML
MIT License
763 stars 222 forks source link

Parser gojson incompatibility with go test's -coverprofile flag #134

Closed forsaken628 closed 1 year ago

forsaken628 commented 2 years ago

When handle yy.txt

Generated report contain incorrect content like this

<testcase name="TestSrv_ServiceMap" classname="example.com/foo/pkg/labels" time="0.000">
    <error message="No test result found"></error>
</testcase>

command: go test -json -race -coverpkg=./... -covermode=atomic -coverprofile=out/cover.out.tmp ./...

go version: go 1.18

jstemmer commented 2 years ago

Thanks. It looks like the testcase gets attributed to the wrong package, possibly because of the order in which it appears in the output.

I don't think I've seen this before, it would be helpful if I could reproduce this output somehow. Can you share a bit more about the environment and the tests that you ran, e.g. what version of Go did you use and did you run any tests in parallel? Did you use any additional commandline options when running go test?

medzin commented 1 year ago

I can confirm that I have a similar issue regardless of whether I'm using a test coverage reporting or not. After disabling the test report in JSON, the tests run stably.

jstemmer commented 1 year ago

I managed to reproduce by testing multiple packages at a time and enabling the race detector (-race). For non-JSON output, test events are always grouped by package. When enabling JSON output this appears to not always be the case, test events from different packages may be mixed occasionally which triggers the bug you observed. Fortunately, each event in the JSON output contains the package name so this is fixable but it will require some refactoring.

Until this has been fixed you can work around this problem by disabling the race detector, or not use the JSON test output.

karelbilek commented 1 year ago

I have been hit by the same bug now. Yes this is fixable by looking at the JSON tag.

I can have a look at this and make a PR

karelbilek commented 1 year ago

Ah, seems like we were using stable 2.0.0 version, which doesn't have this fixed, that's the reason. Nevermind!

altego371 commented 1 year ago

FYI If you are using a version without this fix you can use the next trick: go test -json [...] | jq -c '{k: .Package, v: .}' | sort | jq -c '.v' | go-junit-report [...] where [...] are yours flags It sorts go test output before passing it to the go-junit-report. jq https://stedolan.github.io/ is a cli-tool for json.