jenkinsci / junit-plugin

Allows JUnit-format test results to be published
https://plugins.jenkins.io/junit
MIT License
75 stars 334 forks source link

View full JUnit test output in GitHub checks #492

Open brianjmurrell opened 1 year ago

brianjmurrell commented 1 year ago

What feature do you want to see added?

When I go to the test results in the Checks tab for my PR that were submitted by this plugin, I want to be able to review the full data that was submitted in the JUnit file. Rather I just see a notification that a test failed and a link to go see the full results in Jenkins:

image

There was a ton of information available in the JUnit XML file that that failed result above is reporting. I would like all of that information included in the GitHub Check data so that a user doesn't have to go to Jenkins to see it. I.e. They wouldn't see anything more by going to Jenkins.

Upstream changes

No response

timja commented 1 year ago

The reason is that GitHub limits the character that can be sent.

I'm sure more information could be sent / more intelligently cut test reports to try send more.

brianjmurrell commented 1 year ago

Ultimately, I want to see all of the same detail in the GH Checks reports that I can see in the Jenkins Tests reports.

timja commented 1 year ago

Ultimately, I want to see all of the same detail in the GH Checks reports that I can see in the Jenkins Tests reports.

You can't do that because of character limits in GitHub checks, unfortunately. Stacktraces, stdout, stderr is the main problem, if those weren't sent there would likely be enough characters for a lot more.

brianjmurrell commented 1 year ago

Stacktraces, stdout, stderr is the main problem

But those are exactly the things in a Test report that are useful. Without them I would submit that sending any test results to GH Checks is quite useless as they are completely unactionable.

Has this issue of character limits been opened with GitHub with these kinds of specific use-cases highlighted?

timja commented 1 year ago

It's 65535 characters

Googling 'github checks character limit' will get you many examples and issues. Most of them relate to asking GitHub to document it.

Has this issue of character limits been opened with GitHub with these kinds of specific use-cases highlighted?

Unsure