Can I use only LCOV files?

[moonkwoo]

Hello,

My application is based on python and I use pytest-cov to create report file. The available formats of reports file are terminal, html, xml in pytest-cov. (c.f. https://pytest-cov.readthedocs.io/en/latest/reporting.html)

Can I use not only LCOV files but also XML, HTML to send report file to coveralls?

[ruippeixotog]

I have the same question. I'm trying to migrate from Travis to GitHub Actions and I have a cobertura.xml file generated by sbt-scoverage but I can't seem to find a way to convert it to LCOV. Previously I used sbt-coveralls, but they don't support GitHub Actions and I'd rather use the officially supported action anyway.

Would it be a possibility to support some of the most common coverage formats?

[riquellopes]

Hi guys. I've the same problem with my python application. Do you've a solution to solve this?

[riquellopes]

Hello,

My application is based on python and I use pytest-cov to create report file. The available formats of reports file are terminal, html, xml in pytest-cov. (c.f. https://pytest-cov.readthedocs.io/en/latest/reporting.html)

Can I use not only LCOV files but also XML, HTML to send report file to coveralls?

Hi moonkwoo, to solve I used this tip: https://github.com/aodj/icelandreview/blob/master/.github/workflows/pythonpackage.yml#L30

[Rotendahl]

The solution provided by @riquellopes works. However it does not run on forked pull requests like the official action.

A solution that either converts python-coverage to .lcov or extending the official action to support some other format is still needed.

[SVilgelm]

@nickmerwin Could you give us an example how to use the coverage repo generated by pytest-cov? @coveralls

[SteadBytes]

@SVilgelm that output is not supported by this GitHub action (LCOV only) and requires using the coveralls python library instead which, as @Rotendahl pointed out, doesn't work on forked PRs.

[ruippeixotog]

@nickmerwin are there any plans from Coveralls to officially maintain and/or add new features to this, or was this just an experiment?

[DomAmato]

no update in months and limited functionality... kind of a shame since the github actions are nice but I guess for tests we can't really rely on this

[chrisjsewell]

Oh well looks like I'm ditching coveralls for codecov. It's a shame I like coveralls, but if forked PRs aren't supported with GitHub actions/pytest then its not viable for my, and I imagine lots of others, use cases

[AndreMiras]

I made it work and wrote an action for it https://github.com/marketplace/actions/coveralls-python It's still early so the API may change, always keep an eye on the documentation is something breaks and keep me posted how it works or doesn't work for you

[mikeengland]

We also use Python with Coveralls so this is a blocker for us. Unfortunately we create PRs from forked repos which cannot access the Coveralls token secret, so there is no way to use the manual coveralls module to upload the coverage via Github Actions.

[baztian]

Was considering switching to sonarcloud.io but they don't seem to support matrix builds very well: https://community.sonarsource.com/t/cannot-run-sonarsource-sonarcloud-github-action-master-java-not-found/14922

[SVilgelm]

I moved all my projects to CodeCov, they added tokenless mode for GitHub Actions and it supports any type of coverage reports.

[MarvinT]

For anyone still trying to get pytest working, I think there have been some changes to the env vars needed to run coveralls and get it to find the correct repo on coveralls.io

      - name: Coveralls
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          coveralls

N.B. I have to pip install git+https://github.com/TimoRoth/coveralls-python.git until https://github.com/coveralls-clients/coveralls-python/pull/227 is pulled in.

An example configuration is here: https://github.com/greenelab/ponyo/pull/11 but its complicated a bit by using a conda installation to deal with R dependencies

[casperdcl]

looks like https://github.com/AndreMiras/coveralls-python-action is a viable replacement for coverallsapp/github-action (and https://github.com/AndreMiras/coveralls-python-action/pull/5 will even allow parallel builds).

[nemanjamart]

Apparently coveralls package is now using just GITHUB_TOKEN for the verification of reports. So for upload of reports to the coveralls.io you just need GITHUB_TOKEN defined in that particular step! Considering that GITHUB_TOKEN is automatically created by GitHub to be used in a workflow, it is available to the workflows from the forked repos as well.

Here are versions that I have been using in my projects that are properly executing upload of reports to coveralls from PRs from forked repos. No secrets from original repo necessary!

coveralls==2.2.0
coverage==5.2.1
pytest==5.1.0
pytest-cov==2.7.1

And here is the relevant actions.yml file:

name: GitHub Actions CI

on: [pull_request]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: 3.8

    - name: Install dependencies
      run: |
        pip install -U -r requirements.txt
        pip install -U -r dev-requirements.txt

    - name: Test with pytest
      run: |
        pytest

    - name: Upload coverage data to coveralls.io
      run: coveralls
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[actionless]

in case of running Coveralls inside Docker, you need to bypass there more env vars from Github inside into container:

    GITHUB_ACTIONS=1 \
    GITHUB_TOKEN=$GITHUB_TOKEN \
    GITHUB_RUN_ID=$GITHUB_RUN_ID \
    GITHUB_REF=$GITHUB_REF \

[normanius]

This is just a summary post. I tried several things, and I'm reporting what worked for me.

I was able to link the coverage output with coveralls.io successfully using either the solution by @AndreMiras (see post and github action as well as @nemanjamart's post.

Thanks to this comment by @solancer, I was finally able to get this working also with newer versions of coveralls (3.x). Calling coveralls with the flag --service=github solves some problem. Without the flag, I get: "requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs"

The official coveralls.io github action was not working for me.

As suggested by @nemanjamart, it's a good idea to specify the exact versions of packages. Unfortunately, this ecosystem (github, pytest, coveralls.io, coveralls-github-action) has not been very stable over time.

Below the github workflow specs and the versions of packages that are currently working for me.

coverage==5.5
pytest==6.2.1
pytest-cov==2.11.1
coveralls==2.2.0
# coveralls==3.0.1  # works also

name: Unit tests

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main, develop ]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8, 3.9]

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install pytest
        python -m pip install pytest-cov
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
        python -m pip install -e .
    - name: Test with pytest
      run: |
        pytest --cov

    - name: Upload coverage data to coveralls.io
      run: |
        python -m pip install coveralls==2.2
        coveralls --service=github
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[ltalirz]

in case of running Coveralls inside Docker, you need to bypass there more env vars from Github inside into container:

  GITHUB_ACTIONS=1 \
  GITHUB_TOKEN=$GITHUB_TOKEN \
  GITHUB_RUN_ID=$GITHUB_RUN_ID \
  GITHUB_REF=$GITHUB_REF \

One compact way of accomplishing this (worked for me):

docker exec --tty --env-file <(env | grep GITHUB_) -e GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} $CONTAINER_ID  /bin/bash -l -c 'coveralls --service=github'

[dinkopehar]

So, based on OP question, only way to upload coverage in XML is to convert it to lcov format I guess ?

[afinetooth]

@dinko-pehar the answer is yes, only LCOV. That’s for the Coveralls GitHub action on its own. It only accepts LCOV.

That said, some other integrations—language specific, community-created integrations—support other coverage report formats (typically the standard more common to the language), which those integrations convert to the JSON expected by the Coveralls API. (Which is what the Coveralls GitHub Action does—converts LCOV to JSON.)

if you’re working in python, check out this project: https://github.com/TheKevJames/coveralls-python

(It supports GitHub Actions for CI.)

For other languages, see the Coveralls docs site and look for your language in the left-hand nav: http://docs.coveralls.io

[dinkopehar]

I'm looking into Crystal lang. Using some coverage tools compatible with Crystal, it's able to generate cobertura.xml report. Maybe I can create tool to convert from cobertura to lcov and then upload it. I saw tools which convert lcov to cobertura, so I guess it's possible.

[afinetooth]

@dinko-pehar in that case check out Coveralls’ new Universal Coverage Reporter: https://github.com/coverallsapp/coverage-reporter

It’s written in Crystal and currently has support for LCOV and SimpleCov.

Maybe you can contribute a plug-in adding read-and-convert support for your standard following its model.

[dinkopehar]

That's great. I'll have a look at that and how it can be extended I hope.

[AngellusMortis]

I managed to get everything working nicely using coveragepy-lcov. It requires Python 3.8+ though so if you have an older version, you may need to do a setup-python to get the correct version after running pytest.

Something similar to:

- name: Setup Python
  uses: actions/setup-python@v2
  with:
    python-version: '3.9'

- name: Covert Coverage Results
  run: |
    pip install coveragepy-lcov
    coveragepy-lcov --data_file_path .coverage --output_file_path lcov.info

- name: Upload Coverage Results
  uses: coverallsapp/github-action@master
  with:
    github-token: ${{ secrets.github_token }}
    path-to-lcov: lcov.info

EDIT: As a note, I am not able to get this to report coverage correctly to Coveralls. Coverage is always being reported as 0%. It might be an issue with coveragepy-lcov or it might be an issue with the paths that the Coveralls Github Action is expecting.

[razzeee]

Thanks @AngellusMortis was able to figure it out due to your post https://github.com/trakt/script.trakt/pull/566

[lluc]

EDIT: As a note, I am not able to get this to report coverage correctly to Coveralls. Coverage is always being reported as 0%. It might be an issue with coveragepy-lcov or it might be an issue with the paths that the Coveralls Github Action is expecting.

I also have the same problem : in Coveralls, coverage is always reported as 0% :thinking:

[riquellopes]

The simplest was to do this

- name: Test
      env:
        COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
      run: |
        pytest -s -v --cov={project} --cov-report term-missing && coveralls

[maziyarpanahi]

It would be great to have support for sbt as well. Currently, we do something like:

 - name: Upload coverage data to Coveralls
        run: sbt ++${{ matrix.scala-version }} coverageAggregate coveralls
        env:
          COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          COVERALLS_FLAG_NAME: Scala ${{ matrix.scala-version }}

But it doesn't leave a comment this way. Would be great if we can use the official GA support for sbt.