tebeka / go2xunit

Convert "go test" output to xunit compatible (used in Jenkins/Hudson)
MIT License
162 stars 46 forks source link

Jenkins: Attribute 'skip' is not allowed to appear in element 'testsuite' #60

Closed mpictor closed 6 years ago

mpictor commented 6 years ago

Seen after a jenkins upgrade. Had worked fine before that. Using go1.9.4 and the latest rev of go2xunit (b1260ed1eac).

XUnit plugin is

"jenkins.versions.plugin.xunit": {
"value": "2.0.1"
},

Jenkinsfile:

    //...
    sh """
go test -v ./... >tests.out 2>&1
go2xunit -input tests.out -output test_output.xml
"""
    step([$class: 'XUnitBuilder',
        thresholds: [[$class: 'FailedThreshold', unstableThreshold: '1', failureThreshold: '1']],
        tools: [[$class: 'JUnitType', pattern: 'test_output.xml']]])

Output:

...
INFO: Starting to record.
INFO: Processing JUnit
INFO: [JUnit] - 1 test report file(s) were found with the pattern 'test_output.xml' relative to '/var/build/workspace/job' for the testing framework 'JUnit'.
WARNING: The file '/var/build/workspace/job/test_output.xml' is an invalid file.
WARNING: At line 4 of file:/var/build/workspace/job/test_output.xml:cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'.
...
WARNING: At line 534 of file:/var/build/workspace/job/test_output.xml:cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'.
ERROR: There is at least one problem. Check the Jenkins system log for more information. (if you don't have configured yet the system log before, you have to rebuild).
INFO: Setting the build status to FAILURE
INFO: Stopping recording.
tebeka commented 6 years ago

Thanks! I'll have a look soon-ish. Can you provide the XML file?

mpictor commented 6 years ago

Unfortunately I'm on vacation now and will be hectically busy for a while after. If you still need it in 3-4 weeks, bug me then?

tebeka commented 6 years ago

·Sure, will do.

tebeka commented 6 years ago

Thanks, will try to work on it this weekend.

On Thu, Jun 28, 2018 at 11:12 AM cph1164 notifications@github.com wrote:

Its affecting me too. Here's the XML:

`

` thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tebeka/go2xunit/issues/60#issuecomment-400950941, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFWkV7Kn2h-vo6poL20xN07fEC5c3Rgks5uBI_ZgaJpZM4Up5h2 .

tebeka commented 6 years ago

Actually, I need the input file, not the XML output

Bezbran commented 6 years ago

I run into this issue too, the input file is: ? event_receiver_http/datastructures [no test files] === RUN TestFixCreatedTime --- FAIL: TestFixCreatedTime (0.00s) proccesors_test.go:25: Expected (time.Time, 2018-07-01 08:13:30.057 +0000 UTC). Actual (time.Time, 2018-07-01 08:13:30.056 +0000 UTC) FAIL exit status 1 FAIL event_receiver_http/processors 0.003s

tebeka commented 6 years ago

Thanks @Bezbran , can you post example of the input?

Bezbran commented 6 years ago

Some background: I have script which loop over the directories and run go test -v $(DIR_NAME) >> test_result.txt for each directory. The final file is the input to go2xunit. The content of test_result.txt is my previous comment:

? event_receiver_http/datastructures [no test files] === RUN TestFixCreatedTime --- FAIL: TestFixCreatedTime (0.00s) proccesors_test.go:25: Expected (time.Time, 2018-07-01 08:13:30.057 +0000 UTC). Actual (time.Time, 2018-07-01 8:13:30.056 +0000 UTC) FAIL exit status 1 FAIL event_receiver_http/processors 0.003s

mpictor commented 6 years ago

Note that tests.out has been sanitized via find-replace; this shouldn't affect the output in a way that will matter.

go test -v com/corp/pkg >tests.out 2>&1 cat tests.out

=== RUN   TestFindRequests
--- PASS: TestFindRequests (0.00s)
    request_test.go:32: [74 75]
=== RUN   TestListRequests
--- PASS: TestListRequests (0.00s)
    request_test.go:44: [74 75]
=== RUN   TestList
--- PASS: TestList (0.00s)
=== RUN   TestParseRequestInfo
=== RUN   TestParseRequestInfo/a
=== RUN   TestParseRequestInfo/b
=== RUN   TestParseRequestInfo/c
=== RUN   TestParseRequestInfo/d
--- PASS: TestParseRequestInfo (0.00s)
    --- PASS: TestParseRequestInfo/a (0.00s)
    --- PASS: TestParseRequestInfo/b (0.00s)
    --- PASS: TestParseRequestInfo/c (0.00s)
    --- PASS: TestParseRequestInfo/d (0.00s)
=== RUN   TestSortIdents1
--- PASS: TestSortIdents1 (0.00s)
    sort_ident_test.go:22: idents:
        [blah blah]
        sidents:
        [7221316092118 2482830254660]
    sort_ident_test.go:24: sorted sidents:
        [2482830254660 7221316092118]
    sort_ident_test.go:25: sequential: false
=== RUN   TestSortIdents2
--- PASS: TestSortIdents2 (0.00s)
=== RUN   TestSortIdents3
--- PASS: TestSortIdents3 (0.00s)
=== RUN   TestSortedIdents
--- PASS: TestSortedIdents (0.00s)
=== RUN   TestXpsConfig
--- PASS: TestXpsConfig (0.00s)
PASS
ok      com/corp/pkg    0.003s

go2xunit -input tests.out -output test_output.xml

[Pipeline] step INFO: Starting to record. INFO: Processing JUnit INFO: [JUnit] - 1 test report file(s) were found with the pattern 'test_output.xml' relative to '/var/build/workspace/project' for the testing framework 'JUnit'. WARNING: The file '/var/build/workspace/project/test_output.xml' is an invalid file. WARNING: At line 3 of file:/var/build/workspace/project/test_output.xml:cvc-complex-type.3.2.2: Attribute 'skip' is not allowed to appear in element 'testsuite'. WARNING: The result file '/var/build/workspace/project/test_output.xml' for the metric 'JUnit' is not valid. The result file has been skipped. INFO: Check 'Failed Tests' threshold. [Pipeline] } [Pipeline] // stage [Pipeline] echo Caught: java.lang.NullPointerException

mpictor commented 6 years ago

@tebeka have you had a chance to check what I pasted above, to see if it's enough to pinpoint the issue?

tebeka commented 6 years ago

@mpictor Looking into it, will update. The XML output from go2xunit is a valid XML (ran it though tidy -xml -iq)

tebeka commented 6 years ago

I've created a Jenkins (2.121.1) project with dummy command and told if the JUnix results was the output of go2xunit which I generated from the above output - all worked well.

Any more information I need to know?

jeniks-config jenkins-result

Bezbran commented 6 years ago

Thank you @tebeka I am using Process xUnit test result report plugin for that purpose and it failed to process the files. When i am using Publish JUnit test result report plugin, it is working well.

tebeka commented 6 years ago

Thanks @Bezbran. I'll update the documentation and will look into the differences between JUnit XML and XUnit XML.

mpictor commented 6 years ago

So in spite of the project being named go2xunit it actually only works with JUnit?

tebeka commented 6 years ago

@mpictor The x stands for "any" - we support Jenkins (via JUnit), xunitnet and bamboo. I agree the name is a bit misleading and maybe I should change the default output to XUnit - will think about it.

mpictor commented 6 years ago

I see, that usage didn't occur to me. I'm sure GitHub wouldn't allow a name like go2*unit...

Maybe document that it's not 100% compatible with XUnit, and make sure sites like stack overflow correctly note what it is/isn't compatible with? I think a question on SO recommended go2xunit+XUnit, though it's quite possible that I just didn't read carefully enough.

tebeka commented 6 years ago

I'll work in improving the documentation.