Open tyrelkostyk opened 5 years ago
how did you set the Flask's app config? the coverage would not record when you set the app config debug is True, hope this could help
Also, it seems that celery is using billiard for creating worker processes, and that prevents coverage from getting info on the code that get's run in the workers. According to this
multiproc.py
can be copied almost exactly by replacing multiprocessing with billiard everywhere and removing the env.PYVERSION
check.
I think what might be missing from coverage is an appropriate plugin entrypoint to make use of it, however. I had to use non-public API:
class BilliardCoveragePlugin(CoveragePlugin):
def configure(self, config):
# coverage does not want us to do this, see
# https://github.com/nedbat/coveragepy/blob/6ddd8c8e3abf1321277a07ecd2a5b1c857163ee1/coverage/control.py#L271-L275
if isinstance(config, CoverageConfig):
config_file = config.config_file
elif isinstance(config, Coverage):
config_file = config.config.config_file
else:
raise RuntimeError('Unexpected config type.')
patch_billiard(rcfile=config_file)
def coverage_init(reg, options):
reg.add_configurer(BilliardCoveragePlugin())
Here's the referenced snippet inline: https://github.com/nedbat/coveragepy/blob/6ddd8c8e3abf1321277a07ecd2a5b1c857163ee1/coverage/control.py#L271-L275
I assume this will break in some scenarios since it's not called from _init_for_start
.
@RazerM Can you show me how you used this code? Where is patch_billiard
? It it would help, we can make the config_file available.
@nedbat patch_billiard
is from a copy of coverage/multiproc.py with s/multiprocessing/billiard
.
As far as the original topic of this issue goes, I fear I may have spoken too soon. Celery uses billiard and the above code works for simple uses of billiard.Process
, but I couldn't get it to work with code that runs in celery workers. So I didn't end up using it...
For the benefit of others, I got coverage to work in the workers using something like the following in the tasks file that your celery app imports:
from celery.signals import worker_process_init, worker_process_shutdown
from coverage import Coverage
COV = None
@worker_process_init.connect
def start_coverage(**kwargs):
global COV
COV = Coverage(data_suffix=True)
COV.start()
@worker_process_shutdown.connect
def save_coverage(**kwargs):
if COV is not None:
COV.stop()
COV.save()
Another option may be related to how the Celery pool (at least for the default configuration of "prefork") uses forking to create workers. The use of forking causes several issues including, in my experience, difficulty with gathering coverage data.
See https://github.com/celery/celery/issues/6036#issuecomment-1152424962 for a possible workaround.
I'm running a docker deployed Flask App, and have been testing with the native python unittest library, and monitoring the coverage of said testing with coverage.py. It was working great, up until we integrated some asynchronous tasks using Celery.
When we run these tasks, they pass all of their unit tests. But no matter what we do, the tasks remain "uncovered" in our coverage reports, which we know to be incorrect from our observations. Is there any known bug that would cause this, or a known fix? Any and all help is appreciated!
setUp & tearDown for unittests:
Example Test Case
Brief description of API flow: Make a post to the DB via
api.new_post
, make a request for statistical data throughapi.statistics
which simply calls the Celery taskasync_stats
, which returns it's value tostats_location
(this url gets passed to response from initial api.statistics endpoint, using the celery task.id).simplified API endpoint that triggers async task
And this all works; our actual unit tests are more robust & in-depth than this and it passes every time. Putting in tons of print/logging statements in the
async_stats
task function & monitoring the web/celery logs proves that the asynchronous tasks are being ran, they just refuse to show up on coverage reports. Have seen good results with pytest-cov, but we already have over 200 test cases using unittest & coverage.py. Would rather not have to redo all those...requirements.txt for docker container