Coverage=5.3, Python=2.7, when i send new request to my server and collect the coverage, it does not include my new requests

[Chenyu-dev353]

The background of this issue is that I need to collect the coverage data of running Python project without interrupting the process. However I found that it always only shows the original coverage data for some constants, so I wrote a demo for this and found the same problem.

For some reason we have to use the shell file to create a new bootstrap.py file and start the application:

#!/bin/bash

# create bootstrap.py file
cat << 'EOF' > bootstrap.py
import gc

from coverage import Coverage
import signal
import logging

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
cov = Coverage(source=['/Users/bytedance/Desktop/demo'], data_suffix=True)
cov.erase()
cov.start()
count = 1
def exit_signal_handler(sig=None, frame=None):
    global count
    logging.info("exit signal handler for cov type: {}".format(count))
    count += 1
    global cov
    try:
        logging.info("stop cov")
        cov.stop()
        logging.info("stop cov success")
    except Exception as e:
        logging.error("stop cov fail: {}".format(e))
    try:
        logging.info("save cov")
        cov.save()
        logging.info("save cov success")
    except Exception as e:
        logging.error("save cov fail: {}".format(e))
    try:
        del cov
        gc.collect()
        logging.info("create new cov instance")
        cov = Coverage(source=['/Users/bytedance/Desktop/demo'], data_suffix=True)
        cov.start()
        logging.info("new cov instance started successfully")
    except Exception as e:
        logging.error("new cov instance start fail: {}".format(e))

signal.signal(signal.SIGUSR2, exit_signal_handler)

from flask import Flask

app = Flask(__name__)

from views import *

if __name__ == '__main__':
    app.run(debug=True, threaded=False, processes=10)

EOF

export COVERAGE_PROCESS_START=/Users/bytedance/Desktop/demo/.coveragerc

exec python -m coverage run bootstrap.py

the code above is my bootstrap.py, where I started my Python project, and it also has one useful subfile(views.py) as below:

1. from bootstrap import app
2. import time
3. import time
4. 
5. 
6. @app.route('/home')
7. def home():
8.    return 'Hello, Flask!'
9.
10.
11. @app.route('/demo')
12. def demo():
13.     x = 1
14.     return str(x)
15.
16.
17. @app.route('/test')
18. def test():
19.    x = 1
20.    y = 2
21.    return str(x + y)

The coveragerc file is also attached here:

[run]
parallel = True
concurrency = multiprocessing

[report]
exclude_lines =
    pragma: no cover
    def __repr__
    if self\.debug
    raise AssertionError
    raise NotImplementedError
    if 0:
    if __name__ == .__main__.:
    ^import\s+
    ^from.*import.*

ignore_errors = True
fail_under = 0
skip_covered = False
sort = Cover

The version for Coverage.py and my Python version is 5.3 and 2.7 separately, and here is what i do:

1. Start the project by running bootstrap.sh
2. execute this:

pids=$(ps -ef | grep Python | grep -v grep | awk '{print $2}') 
kill -SIGUSR2 $pids

3. execute: coverage combine && coverage report && mv .coverage .coverage.before (otherwise the original coverage data file will be covered by coverage combine but that's not that important)
4. I found that the missing lines of views.py are line 6-21, which is right because I have excluded all the "import" lines
5. Then I sent a GET request to 127.0.0.1:5000/home, and execute as step 2 and 3
6. the missing lines of views.py stay as line 6-21, which is incorrect.

Please inform me if you have any idea of this problem, as our project is using Python 2.7 and cannot be updated to Python 3 temporarily, Thank you for your help

[nedbat]

You have logging in your coverage management code. Can you show the output you are seeing? Also, are there new subprocesses spawned to handle requests?

[Chenyu-dev353]

After some debugs, I have updated my bootstrap file above, please check before see this.

I am not sure if I have missed some processes or something went wrong in Coverage.py settings, but I will still attach my logs here.

So for the step 4, 6, the logs are as below:

step 4: I started the application, make no request and collect the coverage by sending SIGUSR2 to all processes containing "Python" in its name:

2024-09-19 10:44:00,217 - INFO - exit signal handler for cov type: 1
2024-09-19 10:44:00,217 - INFO - stop cov
2024-09-19 10:44:00,217 - INFO - exit signal handler for cov type: 1
2024-09-19 10:44:00,217 - INFO - stop cov
2024-09-19 10:44:00,217 - INFO - stop cov success
2024-09-19 10:44:00,217 - INFO - save cov
2024-09-19 10:44:00,217 - INFO - stop cov success
2024-09-19 10:44:00,217 - INFO - save cov
2024-09-19 10:44:00,230 - INFO - save cov success
2024-09-19 10:44:00,230 - INFO - create new cov instance
2024-09-19 10:44:00,230 - INFO - save cov success
2024-09-19 10:44:00,230 - INFO - create new cov instance
2024-09-19 10:44:00,234 - INFO - new cov instance started successfully
2024-09-19 10:44:00,234 - INFO - new cov instance started successfully

And for step 6, I sent a request and collect again, the logs are:

2024-09-19 10:45:31,143 - INFO - exit signal handler for cov type: 2
2024-09-19 10:45:31,143 - INFO - stop cov
2024-09-19 10:45:31,143 - INFO - exit signal handler for cov type: 2
2024-09-19 10:45:31,143 - INFO - stop cov
2024-09-19 10:45:31,143 - INFO - stop cov success
2024-09-19 10:45:31,143 - INFO - save cov
2024-09-19 10:45:31,143 - INFO - stop cov success
2024-09-19 10:45:31,143 - INFO - save cov
2024-09-19 10:45:31,153 - INFO - save cov success
2024-09-19 10:45:31,153 - INFO - create new cov instance
2024-09-19 10:45:31,153 - INFO - save cov success
2024-09-19 10:45:31,153 - INFO - create new cov instance
2024-09-19 10:45:31,156 - INFO - new cov instance started successfully
2024-09-19 10:45:31,156 - INFO - new cov instance started successfully

Everything seems to be right but I still cannot collect the coverage, the result is still 0%

[Chenyu-dev353]

It seems that if I use the Flask's method app.run() with multiprocesses, I will not collect any coverage data.

[Chenyu-dev353]

I will appreciate it if you can maybe give us some hint about this problem, and i am not sure if there is some configuration error or it is just because there might be some bug in the version 5.3

[nedbat]

You should read this: https://coverage.readthedocs.io/en/latest/subprocess.html

coverage 5.3 is unsupported and Python 2.7 is unsupported, but that might fix your problem.