pepkit / pipestat

Pipeline results reporting package
https://pep.databio.org/pipestat/
BSD 2-Clause "Simplified" License
4 stars 2 forks source link

move codecov to base coverage report #167

Closed khoroshevskyi closed 3 months ago

khoroshevskyi commented 4 months ago

Sometimes Codecov gives us a hard time and causes errors. I believe we should consider transitioning to using the base pytest coverage report instead. This can be achieved with the GitHub app called Smokeshow: https://github.com/samuelcolvin/smokeshow I've already implemented this approach in a few repositories, and it has been working great. For example, you can check out the configuration in our bbconf repository: e.g. https://github.com/databio/bbconf/blob/master/.github/workflows/cli-coverage.yml

nsheff commented 4 months ago

It's fine with me.

donaldcampbelljr commented 3 months ago

I was able to add your workflow as suggested. Interestingly, I get different results if I run the process locally vs via github actions: image

Digging into this, it looks like our db tests are skipping (for both pytest and this code coverage workflow). I did add the proper dependencies and confirmed the db parameters were correct. image

The check exists within conftest.py:

try:
    subprocess.check_output(
        "docker inspect pipestat_test_db --format '{{.State.Status}}'", shell=True
    )
    SERVICE_UNAVAILABLE = False
except:
    register(print, f"Some tests require a test database. To initiate it, run:\n{DB_CMD}")
    SERVICE_UNAVAILABLE = True
khoroshevskyi commented 3 months ago

Did you set up database correctly?

donaldcampbelljr commented 3 months ago

Nathan and I discussed this issue. We are going to remove coverage via GHA and instead just occasionally run it locally to ensure we are not going too low (threshold ~ 50-60% coverage).

donaldcampbelljr commented 3 months ago

Closing for now as won't fix