Creation of check run fails - 65K limit on Check Run API

[godefroi]

My tests are running successfully, but the creation of the check run is failing:

Adding Check Run

Mapping check status to test outcome...

All tests passed

Invoke-WebRequest: D:\a\_actions\zyborg\dotnet-tests-report\v1.2.0\action.ps1:123
Line |
 123 |      Invoke-WebRequest -Headers $hdr $url -Method Post -Body ($bdy | C …
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"message":"Invalid request.\n\nOnly 65535 characters are allowed; 85289 were
     | supplied.","documentation_url":"https://docs.github.com/rest/reference/checks#create-a-check-run"}

I currently have 862 tests (of which 12 are skipped).

[ebekker]

Unfortunately, this is because the Check Run API has a 65K limit, which can't be overcome.

Instead, if your tests exceed this limit, I suggest you generate the "Gist-based" report using the gist_name and gist_token inputs, and skip the Check Run embedded report using the skip_check_run switch. See here for more details.

I'm closing this for now, but feel free to continue discussion.

[godefroi]

Check Runs can be updated, correct? Would it be possible to split the call into two, first creating the run with half and then updating with the rest? Or is the total size of a check run limited to 65K?

If the check run overall is limited to 65K, then we should be given the option to produce a smaller output, because I don't feel like the number of tests I have here is particularly large.

[ebekker]

I believe it's limit to 65K per run, the text content as a whole. This appears to be a hard-coded limit that others are running into as well, here is one example that's pending some guidance from GH: https://github.com/github/docs/issues/2403

My suspicion is that this won't be corrected any time soon, as the use of Check Runs for report output is "not exactly what they had in mind" for those types of artifacts.

You can try to produce smaller output by updating the XSL transformation to produce smaller content, but ultimately, you'll keep hitting a limit -- it's just the size of the final content, and there's not much to do about that. The Gist option provides a nice alternative as it's durable and easily referenceable such as from a badge, and it's only limited to the max size of a single git object.

[godefroi]

I'm a little nervous about a gist, given that this is a for-profit project in a private repository. If I want to update the XSL, do I need to fork the action myself, or is there a way to provide a custom XSL to the action as provided?

[ebekker]

I just published a new version that has a new optional input trx_xsl_path that allows you to provide your own XSL stylesheet to override the Markdown generation. Here's the detail of the input.

Here's a sample of simpler test report that strips out some sections from the default report stylesheet.

If you want to go down this path, I would recommend, starting with the default stylesheet found here and start pruning it down to what you need.

[godefroi]

That's perfect, thanks!