search

jstemmer / go-junit-report

Convert Go test output to JUnit XML
MIT License
763 stars 222 forks source link

benchmark tests marked as error when no unit test found #174

Open janisz opened 8 months ago

janisz commented 8 months ago

For following test output:

CGO_ENABLED is not 0. Compiling with -linkmode=external
goos: linux
goarch: amd64
pkg: github.com/stackrox/rox/central/administration/events/datastore
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
BenchmarkDatastore_Add_and_Flush
BenchmarkDatastore_Add_and_Flush/add_1000_events_to_the_writer_and_flush_it
BenchmarkDatastore_Add_and_Flush/add_1000_events_to_the_writer_and_flush_it-2                  1      11813052 ns/op     2719272 B/op   32634 allocs/op
BenchmarkDatastore_Add_and_Flush/add_5000_events_to_the_writer_and_flush_it
BenchmarkDatastore_Add_and_Flush/add_5000_events_to_the_writer_and_flush_it-2                  1      99072938 ns/op    16966048 B/op  216477 allocs/op
BenchmarkDatastore_Add_and_Flush/add_10000_events_to_the_writer_and_flush_it
BenchmarkDatastore_Add_and_Flush/add_10000_events_to_the_writer_and_flush_it-2                 1     184645515 ns/op    33935608 B/op  432925 allocs/op
BenchmarkDatastore_Add_and_Flush/add_20000_events_to_the_writer_and_flush_it
BenchmarkDatastore_Add_and_Flush/add_20000_events_to_the_writer_and_flush_it-2                 1     368453326 ns/op    67923280 B/op  865761 allocs/op
PASS

Get:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="5" errors="1">
    <testsuite name="" tests="5" failures="0" errors="1" id="0" time="0.664" timestamp="2023-10-26T18:51:22+02:00">
        <testcase name="BenchmarkDatastore_Add_and_Flush" classname="" time="0.000">
            <error message="No test result found"></error>
        </testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_1000_events_to_the_writer_and_flush_it" classname="" time="0.012"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_5000_events_to_the_writer_and_flush_it" classname="" time="0.099"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_10000_events_to_the_writer_and_flush_it" classname="" time="0.185"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_20000_events_to_the_writer_and_flush_it" classname="" time="0.368"></testcase>
        <system-out><![CDATA[CGO_ENABLED is not 0. Compiling with -linkmode=external
goos: linux
goarch: amd64
pkg: github.com/stackrox/rox/central/administration/events/datastore
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
]]></system-out>
    </testsuite>
</testsuites>

Expect

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="4" errors="0">
    <testsuite name="" tests="4" failures="0" errors="0" id="0" time="0.664" timestamp="2023-10-26T18:51:22+02:00">
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_1000_events_to_the_writer_and_flush_it" classname="" time="0.012"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_5000_events_to_the_writer_and_flush_it" classname="" time="0.099"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_10000_events_to_the_writer_and_flush_it" classname="" time="0.185"></testcase>
        <testcase name="BenchmarkDatastore_Add_and_Flush/add_20000_events_to_the_writer_and_flush_it" classname="" time="0.368"></testcase>
        <system-out><![CDATA[CGO_ENABLED is not 0. Compiling with -linkmode=external
goos: linux
goarch: amd64
pkg: github.com/stackrox/rox/central/administration/events/datastore
cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
]]></system-out>
    </testsuite>
</testsuites>
stxml commented 2 months ago

I've encountered the same problem, and it seems to be caused by running a bench test with subtests and the -v flag.