search

pytest-dev / pytest-cov

Coverage plugin for pytest.
MIT License
1.72k stars 211 forks source link

pytest-cov fails with pytest-xdist and dynamic_context = "test_function" #604

Open masaccio opened 1 year ago

masaccio commented 1 year ago

Summary

I am trying to use pytest-cov fails with pytest-xdist and want to get coverage for each trace function to ultimately see which lines and arcs are covered by which test function (which I'll then use to create a minimal set of tests that generate n% coverage eliminating redundant tests).

I can use a static context with --cov-context=test together with pytest-xdist but when I try dynamic_context = "test_function" I get an internal error.

Reproducer

This works:

git clone git@github.com:masaccio/numbers-parser.git
poetry install
poetry run pytest -n logical tests/test_version.py

The relevant sections of my pyproject.toml are:

[tool.coverage.run]
branch = true
# dynamic_context = "test_function"
omit = ["src/numbers_parser/generated/*.py"]

[tool.coverage.html]
directory = "coverage_html_report"
show_contexts = true

[tool.pytest.ini_options]
addopts = "--cov=src/numbers_parser --cov-report=term-missing:skip-covered --cov-context=test"

Edit pyproject.toml to remove the comment for dynamic_context. I also removed --cov-context=test but that doesn't make any difference in that the internal error still happens.

[tool.coverage.run]
branch = true
dynamic_context = "test_function"
omit = ["src/numbers_parser/generated/*.py"]

Then re-run poetry run pytest -n logical tests/test_version.py and I observe an internal error:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 1173, in _execute
INTERNALERROR>     return self.con.execute(sql, parameters)    # type: ignore[arg-type]
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> sqlite3.OperationalError: no such table: file
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 1178, in _execute
INTERNALERROR>     return self.con.execute(sql, parameters)    # type: ignore[arg-type]
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> sqlite3.OperationalError: no such table: file
INTERNALERROR> 
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pluggy/_hooks.py", line 433, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pluggy/_manager.py", line 112, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pluggy/_callers.py", line 133, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pytest_cov/plugin.py", line 298, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/pytest_cov/engine.py", line 348, in finish
INTERNALERROR>     self.cov.save()
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/control.py", line 757, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>            ^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/control.py", line 838, in get_data
INTERNALERROR>     self._post_save_work()
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/control.py", line 869, in _post_save_work
INTERNALERROR>     self._data.touch_files(paths, plugin_name)
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 615, in touch_files
INTERNALERROR>     self._file_id(filename, add=True)
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 417, in _file_id
INTERNALERROR>     self._file_map[filename] = con.execute_for_rowid(
INTERNALERROR>                                ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 1219, in execute_for_rowid
INTERNALERROR>     with self.execute(sql, parameters) as cur:
INTERNALERROR>   File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 137, in __enter__
INTERNALERROR>     return next(self.gen)
INTERNALERROR>            ^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 1207, in execute
INTERNALERROR>     cur = self._execute(sql, parameters)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jon/Library/Caches/pypoetry/virtualenvs/numbers-parser-xoMXdnO9-py3.11/lib/python3.11/site-packages/coverage/sqldata.py", line 1195, in _execute
INTERNALERROR>     raise DataError(f"Couldn't use data file {self.filename!r}: {msg}") from exc
INTERNALERROR> coverage.exceptions.DataError: Couldn't use data file '/Users/jon/Downloads/numbers-parser/.coverage.Jons-MacBook-Air.93985.543340': no such table: file

My config includes some adopts for pytest but commenting those out and using poetry run pytest -n logical --cov=src/numbers_parser tests/test_version.py alone is sufficient to crash. Playing around with arguments I can see ordering doesn't help but the number of threads does. At least on my test machine:

Versions

% poetry run pip freeze | grep pytest
pytest==7.4.0
pytest-check==1.3.0
pytest-console-scripts==1.4.1
pytest-cov==4.1.0
pytest-profiling==1.7.0
pytest-xdist==3.3.1
% poetry run python --version
Python 3.11.4