Strategy Matrix: Reports are overwritten

[clamattia]

When dispatching jobs to different runners using a strategy matrix:

    strategy:
      matrix:
        runner_type: [A, B]
    runs-on: ["self-hosted", "${{ matrix.runner_type }}"]

The comments of the github-bot are overwritten by the two runners (Can be recovered by looking at the history of the comment).

Is there a way to have a separate report per runner (but also not overwrite comments for repeated action-runs)?

[joonvena]

Sorry for the late reply. Should be doable. I will check this and get back to you.

[joonvena]

So if i understood correctly the report itself is already separate as it's run on different runner but the problem is that when it comments the results it will overwrite the report that the first finished runner did output. Also if it's possible would like to see full example workflow how you are running it currently just to get an idea and have some base to test on :)

[clamattia]

Yes. The workaround is to have a job that depends on the matrix and joins the reports (using robot framework report processing) before pushing.

[clamattia]

I will elaborate a bit more on what I had in mind.

Would be nice:

jobs:
  test:
    strategy:
      matrix:
        runner_type: [A, B]
    runs-on: ["self-hosted", "${{ matrix.runner_type }}"]
    steps:
        # - robot test
        - name: Send result
          if: always()
          uses: joonvena/robotframework-reporter-action@v2.1

A workaround:

jobs:
  test:
    strategy:
      matrix:
        runner_type: [A, B]
    runs-on: ["self-hosted", "${{ matrix.runner_type }}"]
    steps:
        # - robot test
  upload_test:
    needs: [test]
    steps:
        # - merge test results using robot tools
        - name: Send result
          if: always()
          uses: joonvena/robotframework-reporter-action@v2.1

[clamattia]

@joonvena I had an idea. Maybe it could be implemented along the way of https://github.com/marketplace/actions/add-pr-comment . The consumer can set a unique message-id. That way they can make sure, that the different workflows/steps have a unique id (but subsequent actions share it) so that they do not overwrite each other but only overwrite past messages with the same id from past action runs?