Support testing multiple versions using `matrix`

[rlh1994]

I currently run my tests via a matrix run to test multiple versions of python. The way I currently use this action is to only comment if the version was 3.10, but if this doesn't complete first and another version fails first then it means the test of 3.10 is cancelled. I can have it comment for each verison but only if it writes a new comment each time.

It would be really helpful to have the ability to overwrite only specific comments (maybe with some defined ID?) based on which matrix run was completed?

Hoepfully that use case makes sense, if you have any questions let me know!

[MishaKav]

Can you maybe attach some pseudocode for the action or some screenshots? I'm not sure that fully understood you.

[rlh1994]

Sure, so this is currently my action file:

name: Python Unit Tests
on:
  pull_request:
    branches:
      - main
      - 'release/**'

jobs:
  build:
    name: Run Python Tests
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.7", "3.8", "3.9", "3.10"]

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r utils/requirements.txt
    - name: Test with pytest
      run: |
        pip install pytest
        pip install pytest-cov
        pip install uuid
        pytest utils/tests --doctest-modules --junitxml=junit/test-results.xml --cov=utils/functions --cov-report=xml:junit/coverage.xml
    - name: Pytest coverage comment
      uses: MishaKav/pytest-coverage-comment@main
      if: matrix.python-version == '3.10'
      with:
        pytest-xml-coverage-path: junit/coverage.xml
        title: Test Coverage Report
        badge-title: Test Coverage
        hide-badge: false
        hide-report: false
        create-new-comment: false
        hide-comment: false
        report-only-changed-files: false
        remove-link-from-badge: false
        junitxml-path: junit/test-results.xml
        junitxml-title: Python 3.10 Test Results

This created a single comment based on the results of 3.10, but if 3.9 fails first then the run for 3.10 gets cancelled. If I don't use the with condition then it just updates the same comment multiple times. Ideally I would in this case like there to be 4 comments, one for each version that gets updated each time. Hope that helps!

[pglagerweij]

I was running into a similar situation today,, functionality wise, was hoping the Title argument or some other would act as ID of the comment and place multiple comments, based on an ID.

Mainly if you run pytest on different environments/runners within a single workflow. - The multiple-files argument doesn't do the trick in this situation because we build the pytest reports on different containers/runners.

[MishaKav]

@rlh1994 @pglagerweij I added the feature that you asked for and released a new version, now you can pass a matrix-value and it will update the comment for each job, you can try it:

- name: Pytest Coverage Comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    unique-id-for-comment: ${{ matrix.python-version }}

or use a specific version

- name: Pytest Coverage Comment
  uses: MishaKav/pytest-coverage-comment@v1.1.39
  with:
    unique-id-for-comment: ${{ matrix.python-version }}

[rlh1994]

Works perfectly, thank you @MishaKav!

[pglagerweij]

Works perfect indeed, thanks @MishaKav