search

nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.01k stars 433 forks source link

Error when using concurrency on a ThreadingHTTPServer #1506

Open robs-nice99 opened 1 year ago

robs-nice99 commented 1 year ago

Describe the bug I get an error instead of a report when running coverage on the following code.

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? Python 3.8.10

  2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.

-- sys -------------------------------------------------------
               coverage_version: 6.5.0
                coverage_module: /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/__init__.py
                         tracer: -none-
                        CTracer: available
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: .coveragerc
                   configs_read: /home/user.name/python-tests/tests/.coveragerc
                    config_file: /home/user.name/python-tests/tests/.coveragerc
                config_contents: b'[run]\nparallel = True\nbranch = True\nconcurrency = multiprocessing,thread\ndynamic_context = test_function\n\n[html]\nshow_contexts = True'
                      data_file: -none-
                         python: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
                       platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29
                 implementation: CPython
                     executable: /home/user.name/.pyVEnv/py3.8_generic/bin/python
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 383407
                            cwd: /home/user.name/python-tests/tests
                           path: /home/user.name/.pyVEnv/py3.8_generic/bin
                                 /usr/lib/python38.zip
                                 /usr/lib/python3.8
                                 /usr/lib/python3.8/lib-dynload
                                 /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages
                                 /home/user.name/.local/lib/python3.8/site-packages
                                 /usr/lib/python3.8/site-packages
                                 /usr/local/lib/python3.8/dist-packages
                                 /usr/lib/python3/dist-packages
                    environment: FDZ_PYPI_TOKEN = ***
                                 FDZ_PYPI_USERNAME = __token__
                                 HOME = /home/user.name
                   command_line: /home/user.name/.pyVEnv/py3.8_generic/bin/coverage debug sys
                sqlite3_version: 2.6.0
         sqlite3_sqlite_version: 3.31.1
             sqlite3_temp_store: 0
        sqlite3_compile_options: COMPILER=gcc-9.4.0, ENABLE_COLUMN_METADATA, ENABLE_DBSTAT_VTAB,
                                 ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS3_TOKENIZER, ENABLE_FTS4,
                                 ENABLE_FTS5, ENABLE_JSON1, ENABLE_LOAD_EXTENSION, ENABLE_PREUPDATE_HOOK,
                                 ENABLE_RTREE, ENABLE_SESSION, ENABLE_STMTVTAB, ENABLE_UNLOCK_NOTIFY,
                                 ENABLE_UPDATE_DELETE_LIMIT, HAVE_ISNAN, LIKE_DOESNT_MATCH_BLOBS,
                                 MAX_SCHEMA_RETRY=25, MAX_VARIABLE_NUMBER=250000, OMIT_LOOKASIDE,
                                 SECURE_DELETE, SOUNDEX, TEMP_STORE=1, THREADSAFE=1, USE_URI
  1. What versions of what packages do you have installed? The output of pip freeze is helpful.
anyio==3.6.2
appdirs==1.4.3
apturl==0.5.2
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.1.0
attrs==22.1.0
autokey==0.95.10
Babel==2.11.0
backcall==0.2.0
bcrypt==3.1.7
beautifulsoup4==4.11.1
bleach==5.0.1
blinker==1.4
Brlapi==0.7.0
certifi==2019.11.28
cffi==1.15.1
chardet==3.0.4
chrome-gnome-shell==0.0.0
Click==7.0
colorama==0.4.3
command-not-found==0.3
coverage==6.5.0
cryptography==2.8
cupshelpers==1.0
dbus-python==1.2.16
debugpy==1.6.3
decorator==5.1.1
defer==1.0.6
defusedxml==0.7.1
distlib==0.3.0
distro==1.4.0
distro-info===0.23ubuntu1
duplicity==0.8.12.0
entrypoints==0.3
exceptiongroup==1.0.4
executing==1.2.0
Faker==9.8.1
fasteners==0.14.1
fastjsonschema==2.16.2
filelock==3.0.12
future==0.18.2
galternatives==1.0.6
httplib2==0.14.0
idna==2.8
importlib-metadata==5.0.0
importlib-resources==5.10.0
iniconfig==1.1.1
ipykernel==6.4.2
ipython==7.34.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
json5==0.9.10
jsonschema==4.17.0
jupyter-server==1.23.1
jupyter_client==7.4.5
jupyter_core==5.0.0
jupyterlab==3.5.0
jupyterlab-pygments==0.2.2
jupyterlab_server==2.16.3
keyring==18.0.1
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
lockfile==0.12.2
louis==3.12.0
macaroonbakery==1.3.1
Mako==1.1.0
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
meld==3.20.2
mistune==2.0.4
monotonic==1.5
more-itertools==4.2.0
nbclassic==0.4.8
nbclient==0.7.0
nbconvert==7.2.4
nbformat==5.7.0
nest-asyncio==1.5.6
netifaces==0.10.4
notebook==6.5.2
notebook_shim==0.2.2
oauthlib==3.1.0
olefile==0.46
packaging==21.3
pandocfilters==1.5.0
paramiko==2.6.0
parso==0.8.3
pexpect==4.6.0
pickleshare==0.7.5
Pillow==7.0.0
pipenv==11.9.0
pkgutil_resolve_name==1.3.10
platformdirs==2.5.3
pluggy==1.0.0
prometheus-client==0.15.0
prompt-toolkit==3.0.32
protobuf==3.6.1
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pycairo==1.16.2
pycparser==2.21
pycups==1.9.73
Pygments==2.13.0
PyGObject==3.36.0
pyinotify==0.9.6
PyJWT==1.7.1
pymacaroons==0.13.0
PyNaCl==1.3.0
pyparsing==3.0.9
pyRFC3339==1.1
pyrsistent==0.19.2
pytest==7.2.0
python-apt==2.0.0+ubuntu0.20.4.8
python-dateutil==2.8.2
python-debian===0.1.36ubuntu1
python-xlib==0.23
pytz==2019.3
pyxdg==0.26
PyYAML==5.4
pyzmq==24.0.1
reportlab==3.5.34
requests==2.22.0
requests-unixsocket==0.2.0
SecretStorage==2.3.1
Send2Trash==1.8.0
simplejson==3.16.0
six==1.14.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.6.0
systemd-python==234
terminado==0.17.0
text-unidecode==1.3
tinycss2==1.2.1
tomli==2.0.1
tornado==6.2
traitlets==5.5.0
ubuntu-advantage-tools==27.12
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
urllib3==1.25.8
usb-creator==0.3.7
virtualenv==20.0.17
virtualenv-clone==0.3.0
wadllib==1.3.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.4.2
xkit==0.0.0
zipp==1.0.0
  1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

test_main.py

import os
import signal
import time
from contextlib import contextmanager
from functools import partial
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer, SimpleHTTPRequestHandler
from multiprocessing import Process
from threading import Thread
import coverage

@contextmanager
def http_server(host: str, port: int, directory: str):
    server = ThreadingHTTPServer(
        (host, port), partial(SimpleHTTPRequestHandler, directory=directory)
    )
    server_thread = Thread(target=server.serve_forever, name="http_server")
    server_thread.start()

    try:
        yield
    finally:
        server.shutdown()
        server_thread.join()

def test_main():

    # setup
    coverage.process_startup()
    with http_server("127.0.0.1", 8087, "."):
        # now you can use the web server
        time.sleep(1)

    # test
    a = 1
    assert a == 1

.coveragerc

[run]
parallel = True
branch = True
concurrency = multiprocessing,thread
dynamic_context = test_function

[html]
show_contexts = True
  1. What commands did you run?
export COVERAGE_PROCESS_START=$PWD/.coveragerc
coverage run -m pytest test_main.py

Expected behavior

Get a coverage report

What I got

Error message

===================================================================================================== test session starts =====================================================================================================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/user.name/python-tests/tests
plugins: anyio-3.6.2, Faker-9.8.1
collected 1 item                                                                                                                                                                                                              

test_main.py .                                                                                                                                                                                                          [100%]

====================================================================================================== 1 passed in 1.08s ======================================================================================================
self._collectors:
  <Collector at 0x7fb2e198fa60: CTracer>
                      <module> : /home/user.name/.pyVEnv/py3.8_generic/bin/coverage:8
                          main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:943
                  command_line : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:659
                        do_run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:827
                         start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:580
               _init_for_start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:487
                      __init__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py:114
  <Collector at 0x7fb2e021e940: CTracer>
                      <module> : /home/user.name/.pyVEnv/py3.8_generic/bin/coverage:8
                          main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:943
                  command_line : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:659
                        do_run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:830
                           run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/execfile.py:199
                      <module> : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pytest/__main__.py:5
                  console_main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/config/__init__.py:190
                          main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/config/__init__.py:167
                      __call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
                     _hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
                    _multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
           pytest_cmdline_main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:317
                  wrap_session : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:270
                         _main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:324
                      __call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
                     _hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
                    _multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
            pytest_runtestloop : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:349
                      __call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
                     _hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
                    _multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
       pytest_runtest_protocol : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:112
               runtestprotocol : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:131
               call_and_report : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:220
             call_runtest_hook : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:259
                     from_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:339
                      <lambda> : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:260
                      __call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
                     _hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
                    _multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
           pytest_runtest_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:167
                       runtest : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/python.py:1789
                      __call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
                     _hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
                    _multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
            pytest_pyfunc_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/python.py:195
                     test_main : /home/user.name/python-tests/tests/test_main.py:34
               process_startup : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:1223
                         start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:580
               _init_for_start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:487
                      __init__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py:114
Traceback (most recent call last):
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 830, in do_run
    runner.run()
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/execfile.py", line 199, in run
    exec(code, main_mod.__dict__)
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
SystemExit: ExitCode.OK

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user.name/.pyVEnv/py3.8_generic/bin/coverage", line 8, in <module>
    sys.exit(main())
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 943, in main
    status = CoverageScript().command_line(argv)
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 659, in command_line
    return self.do_run(options, args)
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 835, in do_run
    self.coverage.stop()
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py", line 604, in stop
    self._collector.stop()
  File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py", line 351, in stop
    assert self._collectors[-1] is self, (
AssertionError: Expected current collector to be <Collector at 0x7fb2e198fa60: CTracer>, but it's <Collector at 0x7fb2e021e940: CTracer>
robs-nice99 commented 1 year ago

Just to give some context, the idea is to use requests to call to the server (a custom one, not the standard SimpleHTTP) instead of the

# test
    a = 1
    assert a == 1

And then check the coverage on that custom server. I get a different error with that code, but while creating a minimum example I got this error and thought I should start here.

nedbat commented 1 year ago

@robs-nice99 Sorry I haven't gotten to this yet, but: it looks like you've exposed a secret in your FDZ_PYPI_TOKEN environment variable above. Sorry about that.

nedbat commented 1 year ago

I've written an issue about coverage debug sys revealing secrets: #1628