search

celery / celery

Distributed Task Queue (development branch)
https://docs.celeryq.dev
Other
24.02k stars 4.62k forks source link

Code coverage is inaccurate #4869

Open johnarnold opened 6 years ago

johnarnold commented 6 years ago

If you look at the Code Coverage results for a PR you can see that function definition lines, decorators and other lines of code that are evaluated on import do not show up in the code coverage even if the functions are called in tests, because coverage starts too late in the run.

this is explained in the coverage faq: http://coverage.readthedocs.io/en/latest/faq.html As well as in https://github.com/pytest-dev/pytest-cov/issues/188

I tried the workaround of modifying tox.ini to run pytest from coverage.py, as suggested in the FAQ and various posts:

 commands =
-    unit: py.test -xv --cov=celery --cov-report=xml --cov-report term
-    integration: py.test -xsv t/integration
+    unit: coverage run {envbindir}/py.test -xv --cov=celery --cov-report=xml --cov-report term
+    integration: coverage run {envbindir}/py.test -xsv t/integration

However, this actually resulted in a pretty drastic DROP in coverage, not the expected increase.

Someone with more experience with this test harness may have better luck figuring this out.

johnarnold commented 6 years ago

Also: @thedrow opened https://github.com/pytest-dev/pytest-cov/issues/184