Annotations not being added to files when there are more than 50 failures

[nfelt14]

I have a PR that I am manipulating to force more than 50 failures on. Not all files that should have annotations are getting them.

https://github.com/nfelt14/tm_devices/pull/10

My settings are here:

      - name: Publish Test Results
        id: test-results
        uses: mikepenz/action-junit-report@v4
        with:
          report_paths: artifacts/**/.results_tests/results.xml
          check_name: Test Results
          require_tests: true
          update_check: true
          check_annotations: true
          annotate_only: true
          job_summary: true

These are the permissions I am using:

    permissions:
      checks: write
      pull-requests: write
      actions: read
      contents: read

[nfelt14]

It appears that the job summary page is only showing 10 annotations, and those annotations are the only ones showing up on the changed files page. When I reduce the number of failures, all of the annotations appear correctly.

[mikepenz]

Practically with the annotations_only flag you will go into this condition: https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L36-L55

It limiting it to 50 in that case sounds like a hard limit from GitHub, given there shouldn't be anything in the action limiting it here.

For example there is a hard limit of 50 annotations when creating a check: https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L81-L100 As far as I remember it would cause problems for this one having a higher number in the past.

[nfelt14]

What I am curious about is why only 10 failures are being shown. The unique failure count is over 10, but less than 50, yet not all annotations are created.

[mikepenz]

Maybe you can run with debug enabled, so you should see in more details which requests are being made.

[nfelt14]

Maybe you can run with debug enabled, so you should see in more details which requests are being made.

I ran with debug mode here: https://github.com/nfelt14/tm_devices/actions/runs/8486298385/job/23306825707?pr=10

I don't see anything indicating requests that are being made.

[mikepenz]

Sorry for the delayed anwer. Looking at the log shows that all error annotations are called by the plugin. I believe the 10 might then be a github limit in the UI

This log comes from here: https://github.com/mikepenz/action-junit-report/blob/main/src/annotator.ts#L48

You can see them when expanding the publish result group

[nfelt14]

Yeah, it seems like it is a GitHub UI limitation. Because of this, I ended up going a simpler route for reporting test results that doesn't use any annotations.