Pytest Coverage Comment

This action comments a pull request or commit with a HTML test coverage report. The report is based on the coverage report generated by your test runner. Note that this action does not run any tests, but expects the tests to have been run by another action already (support pytest only).

Similar Action for Jest

After I see that action become popular, I made similar action (even better) for javascript/typescript that runs jest jest-coverage-comment

You can add this action to your GitHub workflow for Ubuntu runners (e.g. runs-on: ubuntu-latest) as follows:

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

Inputs

Output Variables

notSuccessTestInfo

the format will be JSON.stringify in current structure

{
  "failures": [{ "classname": "...", "name": "..." }],
  "errors": [{ "classname": "...", "name": "..." }],
  "skipped": [{ "classname": "...", "name": "..." }],
  "count": 3
}

Output example

Example usage

The following is an example GitHub Action workflow that uses the Pytest Coverage Comment to extract the coverage report to comment at pull request:

# This workflow will install dependencies, create coverage tests and run Pytest Coverage Comment
# For more information see: https://github.com/MishaKav/pytest-coverage-comment/
name: pytest-coverage-comment
on:
  pull_request:
    branches:
      - '*'

# https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
# `contents` is for permission to the contents of the repository.
# `pull-requests` is for permission to pull request
permissions:
  contents: write
  checks: write
  pull-requests: write

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python 3.8
        uses: actions/setup-python@v2
        with:
          python-version: 3.8

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install flake8 pytest pytest-cov
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      - name: Build coverage file
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=app tests/ | tee pytest-coverage.txt

      - name: Pytest coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml

Example GitHub Action workflow that uses coverage percentage as output (see the live workflow)

- name: Pytest coverage comment
  id: coverageComment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

- name: Check the output coverage
  run: |
    echo "Coverage Percentage - ${{ steps.coverageComment.outputs.coverage }}"
    echo "Coverage Color - ${{ steps.coverageComment.outputs.color }}"
    echo "Coverage Html - ${{ steps.coverageComment.outputs.coverageHtml }}"
    echo "Summary Report - ${{ steps.coverageComment.outputs.summaryReport }}"

    echo "Coverage Warnings - ${{ steps.coverageComment.outputs.warnings }}"

    echo "Coverage Errors - ${{ steps.coverageComment.outputs.errors }}"
    echo "Coverage Failures - ${{ steps.coverageComment.outputs.failures }}"
    echo "Coverage Skipped - ${{ steps.coverageComment.outputs.skipped }}"
    echo "Coverage Tests - ${{ steps.coverageComment.outputs.tests }}"
    echo "Coverage Time - ${{ steps.coverageComment.outputs.time }}"
    echo "Not Success Test Info - ${{ steps.coverageComment.outputs.notSuccessTestInfo }}"

Example GitHub Action workflow that get coverage report from coverage-xml instead of coverage.txt pytest --cov-report "xml:coverage.xml" --cov=src tests/

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-xml-coverage-path: ./coverage.xml

Example GitHub Action workflow that passes all params to Pytest Coverage Comment

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./path-to-file/pytest-coverage.txt
    pytest-xml-coverage-path: ./path-to-file/coverage.xml
    title: My Coverage Report Title
    badge-title: My Badge Coverage Title
    hide-badge: false
    hide-report: false
    create-new-comment: false
    hide-comment: false
    report-only-changed-files: false
    remove-link-from-badge: false
    unique-id-for-comment: python3.8
    junitxml-path: ./path-to-file/pytest.xml
    junitxml-title: My JUnit Xml Summary Title

Example GitHub Action workflow that runs pytest inside docker It will generate pytest-coverage.txt and pytest.xml in /tmp directory inside docker and share /tmp directory with GitHub workspace.

- name: Run unit tests (pytest)
  run: |
    docker run -v /tmp:/tmp $IMAGE_TAG python3 -m pytest --cov-report=term-missing:skip-covered --junitxml=/tmp/pytest.xml --cov=src tests/ | tee /tmp/pytest-coverage.txt

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: /tmp/pytest-coverage.txt
    junitxml-path: /tmp/pytest.xml

Example GitHub Action workflow that uses multiple files mode (see the live workflow)

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    multiple-files: |
      My Title 1, ./data/pytest-coverage_3.txt, ./data/pytest_junit_1.xml
      My Title 2, ./data/pytest-coverage_4.txt, ./data/pytest_junit_2.xml

Example GitHub Action workflow that will update your README.md with coverage report, only on merge to main branch (see the update-coverage-on-readme workflow) All you need is to add in your README.md the following lines wherever you want. If your coverage html report will not change, it wouldn't push any changes to readme file.

<!-- Pytest Coverage Comment:Begin -->
<!-- Pytest Coverage Comment:End -->

name: Update Coverage on Readme
on:
  push:

# https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
# `contents` is for permission to the contents of the repository.
# `pull-requests` is for permission to pull request
permissions:
  contents: write
  checks: write
  pull-requests: write

jobs:
  update-coverage-on-readme:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
          fetch-depth: 0 # otherwise, you will failed to push refs to dest repo

      - name: Pytest coverage comment
        if: ${{ github.ref == 'refs/heads/main' }}
        id: coverageComment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          hide-comment: true
          pytest-coverage-path: ./data/pytest-coverage_4.txt

      - name: Update Readme with Coverage Html
        if: ${{ github.ref == 'refs/heads/main' }}
        run: |
          sed -i '/<!-- Pytest Coverage Comment:Begin -->/,/<!-- Pytest Coverage Comment:End -->/c\<!-- Pytest Coverage Comment:Begin -->\n\${{ steps.coverageComment.outputs.coverageHtml }}\n<!-- Pytest Coverage Comment:End -->' ./README.md

      - name: Commit & Push changes to Readme
        if: ${{ github.ref == 'refs/heads/main' }}
        uses: actions-js/push@master
        with:
          message: Update coverage on Readme
          github_token: ${{ secrets.GITHUB_TOKEN }}

Result example

Collapsed comment

Expanded comment

Multiple Files Mode (can be useful on mono-repo projects)

Badge Colors

Badge	Range
	0 - 40
	40 - 60
	60 - 80
	80 - 90
	90 - 100

Auto updating badge on README

If you want auto-update the coverage badge on your Readme, you can see the workflow

🤝 Contributing

We welcome all contributions. You can submit any ideas as pull requests or as GitHub issues and have a good time! :)

Our Contibutors