search

jstemmer / go-junit-report

Convert Go test output to JUnit XML
MIT License
763 stars 222 forks source link

`system-out` is never placed under the correct `testcase` #175

Open epot opened 8 months ago

epot commented 8 months ago

I am trying to leverage this tool to send my go ci reports and logs to Datadog.

Sadly, it seems that no matter what options I try, the system-out is always put at the testsuite level, for instance:

    <testsuite name="github.com/xxx/pkg/xx" tests="1" failures="1" errors="0" id="75" hostname="stuff.local" time="5.499" timestamp="2023-11-02T09:50:29+01:00">
        <testcase name="TestDeleteSomething" classname="github.com/x/pkg/xx" time="0.000">
            <failure message="Failed"><![CDATA[    controller_test.go:289: 
            Error Trace:    /Users/epot/ws/xxx/pkg/xx/controller_test.go:289
            Error:          Should be true
            Test:           TestDeleteSomething]]></failure>
        </testcase>
        <system-out><![CDATA[2023/11/02 09:50:24 maxprocs: Leaving GOMAXPROCS=12: CPU quota undefined]]></system-out>
    </testsuite>

Is it something not supported?

epot commented 8 months ago

And as a side note (not sure if I should open a separate issue), when the test panics, all the logs is placed under error.message, like:

            <error message="No test result found"><![CDATA[
{"time":"2023-11-02T09:58:14.944416+01:00","level":"INFO","msg":"heartbeat state","git_commit":"","git_time":"0001-01-01T00:00:00Z","git_modified":false,"go_os":"","go_arch":"","process_generation":"68cbe19f-f21e-49aa-b3a7-3c1d9b991409","hostname_fqdn":"","hostname_short":"Emmanuels-MacBook-Pro.local","private_ips":["192.168.1.55","192.168.1.94"],"num_vcpus":12,"server_id":"DNS_SERVER_ID","address":"127.0.0.1:9093","module":"environment","sub_service":"environment","num_actors":0,"used_memory":0}
{"time":"2023-11-02T09:58:14.944427+01:00","level":"INFO","msg":"performed initial heartbeat","git_commit":"","git_time":"0001-01-01T00:00:00Z","git_modified":false,"go_os":"","go_arch":"","process_generation":"68cbe19f-f21e-49aa-b3a7-3c1d9b991409","hostname_fqdn":"","hostname_short":"Emmanuels-MacBook-Pro.local","private_ips":["192.168.1.55","192.168.1.94"],"num_vcpus":12,"server_id":"DNS_SERVER_ID","address":"127.0.0.1:9093","address":"127.0.0.1:9093"}
{"time":"2023-11-02T09:58:14.944475+01:00","level":"INFO","msg":"trying to connect to sqlite","git_commit":"","git_time":"0001-01-01T00:00:00Z","git_modified":false,"go_os":"","go_arch":"","process_generation":"68cbe19f-f21e-49aa-b3a7-3c1d9b991409","hostname_fqdn":"","hostname_short":"Emmanuels-MacBook-Pro.local","private_ips":["192.168.1.55","192.168.1.94"],"num_vcpus":12}
{"time":"2023-11-02T09:58:14.944671+01:00","level":"INFO","msg":"done connecting to sqlite","git_commit":"","git_time":"0001-01-01T00:00:00Z","git_modified":false,"go_os":"","go_arch":"","process_generation":"68cbe19f-f21e-49aa-b3a7-3c1d9b991409","hostname_fqdn":"","hostname_short":"Emmanuels-MacBook-Pro.local","private_ips":["192.168.1.55","192.168.1.94"],"num_vcpus":12}
....
dkirrane commented 3 months ago

I have similar issue:

Reproduce:

  1. Sample test

    func TestPanic(t *testing.T) {
// cause a panic in order to get the test to exit with an exception
panic("This test has a panic")
}

  2. Run go test

    go test -count=1 -v -timeout 1h | tee test_output.log

  3. Will produce test_output.log like,..

=== RUN   TestPanicTerraform
--- FAIL: TestPanicTerraform (0.00s)
panic: This test has a panic [recovered]
    panic: This test has a panic

goroutine 13 [running]:
testing.tRunner.func1.2({0x1877a80, 0x201da60})
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1548 +0x397
panic({0x1877a80?, 0x201da60?})
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/runtime/panic.go:914 +0x21f
github.com/avaya/terratest-action.TestPanicTerraform(0xc000581380?)
    /mnt/c/dev/GitHub/terratest-action/test/terraform_hello_world_test.go:117 +0x25
testing.tRunner(0xc000642680, 0x1c7e1b8)
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x3ad
exit status 2
FAIL    github.com/company/test 0.066s
  1. Create report.xml 
    go-junit-report < test_output.log > report.xml

will produce

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="6" errors="1" failures="5">
    <testsuite name="github.com/company/test" tests="6" failures="5" errors="1" id="0" hostname="foo" time="0.066" timestamp="2024-03-26T18:09:14Z">
        <testcase name="TestPanicTerraform" classname="github.com/company/test" time="0.000">
            <failure message="Failed"></failure>
        </testcase>
        <system-out><![CDATA[panic: This test has a panic [recovered]
    panic: This test has a panic

goroutine 13 [running]:
testing.tRunner.func1.2({0x1877a80, 0x201da60})
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1548 +0x397
panic({0x1877a80?, 0x201da60?})
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/runtime/panic.go:914 +0x21f
github.com/company/test.TestPanicTerraform(0xc000581380?)
    /mnt/c/dev/GitHub/terratest-action/test/terraform_hello_world_test.go:117 +0x25
testing.tRunner(0xc000642680, 0x1c7e1b8)
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
    /home/linuxbrew/.linuxbrew/Cellar/go/1.21.5/libexec/src/testing/testing.go:1648 +0x3ad
exit status 2]]></system-out>
    </testsuite>
</testsuites>