Coverage data from Flask application

[brycefisher]

Describe the bug I'm trying to generate test code coverage reports for a Flask application. I'm doing "blackbox" style testing. The problem is that I'm unable to get coverage.py to produce coverage data (.coverage) files during a test run.

Here's a minimal test case as a small repository: https://github.com/brycefisher/coveragepy-flask

Here's a recent CircleCI run which demonstrates the problem: https://circleci.com/gh/brycefisher/coveragepy-flask/13

I suspect that something about how the Flask development server dispatches requests (to threads) is causing trouble for coverage.py.

To Reproduce How can we reproduce the problem? Please be specific.

1. What version of Python are you running?
2. What versions of what packages do you have installed? The output of pip freeze is very helpful.
3. What code are you running? Give us a specific commit of a specific repo that we can check out.
4. What commands did you run?

Use Python 3.7.0

git clone https://github.com/brycefisher/coveragepy-flask
python -m venv venv
. venv/bin/activate

Run the same commands listed that as on CircleCI: https://github.com/brycefisher/coveragepy-flask/blob/master/.circleci/config.yml#L42-L62

No .coverage file is generated, so coverage xml fails.

Expected behavior A coverage.xml ought to be created which shows that all lines in hello.py are covered

Additional context The closest issue I could find was: #636, however that issue was closed due to complexity arising from uWSGI. This simply uses the most bare bones Flask app possible.

I'll be opening a parallel issue in the Flask project to gather any insights into concurrency related or settrace issues preventing coverage.py from producing data in my minimal-reproduction.

[brycefisher]

Could it be that I need to register a handler for a signal to ensure a coverage data file is written? https://coverage.readthedocs.io/en/v4.5.x/subprocess.html#signal-handlers-and-atexit

[nedbat]

With coverage 4.x, you would need to make sure the coverage data file is written. Coverage.py does that with an atexit handler. In 5.x, the data file should be written as the program progresses, so you might not have this issue any more.

[nedbat]

@brycefisher Have you had a chance to try coverage.py 5.x?

[brycefisher]

I didn't try that yet! I'll give that a shot. It looks like registering the signal handler for SIGTERM was the trick that allowed coverage.py to output the coverage file: https://github.com/brycefisher/coveragepy-flask/commit/870f5e2e6d0f111279482ed6e9e1cbc2b9862a0b#diff-8d35108ffe2ad173a697734a3e9938e1