search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.88k stars 2.65k forks source link

Pytest sometimes fails with the INTERNALERROR message #11765

Open yurivict opened 8 months ago

yurivict commented 8 months ago

Pytest fails with the INTERNALERROR like in the log below on many projects. Pytest also succeeds on other projects.

===>   py39-laspy-2.5.2 depends on file: /usr/local/bin/python3.9 - found
cd /usr/ports/misc/py-laspy/work-py39/laspy-2.5.2 && /usr/bin/env XDG_DATA_HOME=/usr/ports/misc/py-laspy/work-py39  XDG_CONFIG_HOME=/usr/ports/misc/py-laspy/work-py39  XDG_CACHE_HOME=/usr/ports/misc/py-laspy/work-py39/.cache  HOME=/usr/ports/misc/py-laspy/work-py39 PATH=/usr/local/libexec/ccache:/usr/ports/misc/py-laspy/work-py39/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/misc/py-laspy/work-py39/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  "  MANPREFIX="/usr/local" CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" PYTHONPATH=/usr/ports/misc/py-laspy/work-py39/stage/usr/local/lib/python3.9/site-packages /usr/local/bin/python3.9 -m pytest -k '' -rs -v -o addopts= 
==================================================================================== test session starts ====================================================================================
platform freebsd14 -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0 -- /usr/local/bin/python3.9
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/usr/ports/misc/py-laspy/work-py39/laspy-2.5.2/.hypothesis/examples'))
Using --randomly-seed=1862560085
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /usr/ports/misc/py-laspy/work-py39/laspy-2.5.2
plugins: anyio-4.1.0, hypothesis-6.92.0, asyncio-0.22.0, trio-0.8.0, cov-4.1.0, rerunfailures-11.1.2, randomly-3.12.0, timeout-2.1.0, time-machine-2.11.0, forked-1.6.0, flaky-3.7.0, aspectlib-2.0.0, xdist-3.5.0, env-0.6.2, mock-3.10.0, tornado-0.8.1, pytest_httpserver-1.0.8, isort-3.1.0, flake8-1.1.1, subprocess-1.5.0, datadir-1.5.0, benchmark-4.0.0
asyncio: mode=strict
collected 748 items                                                                                                                                                                         
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/ports/misc/py-laspy/work-py39/laspy-2.5.2/tests/cli/__init__.py", line 6, in skip_if_cli_deps_are_not_installed
INTERNALERROR>     from laspy.cli.core import app
INTERNALERROR> ModuleNotFoundError: No module named 'laspy.cli.core'
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 335, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 675, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 845, in genitems
INTERNALERROR>     yield from self.genitems(subnode)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 842, in genitems
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 546, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/pytest_asyncio/plugin.py", line 552, in pytest_collectstart
INTERNALERROR>     marks = get_unpacked_marks(collector.obj, consider_mro=True)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 310, in obj
INTERNALERROR>     self._obj = obj = self._getobj()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 528, in _getobj
INTERNALERROR>     return self._importtestmodule()
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 617, in _importtestmodule
INTERNALERROR>     mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/pathlib.py", line 567, in import_path
INTERNALERROR>     importlib.import_module(module_name)
INTERNALERROR>   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
INTERNALERROR>     return _bootstrap._gcd_import(name[level:], package, level)
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/assertion/rewrite.py", line 186, in exec_module
INTERNALERROR>     exec(co, module.__dict__)
INTERNALERROR>   File "/usr/ports/misc/py-laspy/work-py39/laspy-2.5.2/tests/cli/test_compress_decompress.py", line 9, in <module>
INTERNALERROR>     skip_if_cli_deps_are_not_installed()
INTERNALERROR>   File "/usr/ports/misc/py-laspy/work-py39/laspy-2.5.2/tests/cli/__init__.py", line 8, in skip_if_cli_deps_are_not_installed
INTERNALERROR>     pytest.skip("skipping cli test (deps not installed)", allow_module_level=True)
INTERNALERROR>   File "/usr/local/lib/python3.9/site-packages/_pytest/outcomes.py", line 179, in skip
INTERNALERROR>     raise Skipped(msg=reason, allow_module_level=allow_module_level)
INTERNALERROR> Skipped: skipping cli test (deps not installed)

=================================================================================== no tests ran in 0.50s ===================================================================================
*** Error code 3

What is wrong?

Version: 2.5.2 Python-3.9 FreeBSD 14.0

bluetech commented 8 months ago

The minimal reproduction is:

# test_it.py
import pytest
pytest.skip("skipping cli test (deps not installed)", allow_module_level=True)
# conftest.py
import pytest

def pytest_collectstart(collector):
    if type(collector) is pytest.Module:
        collector.obj

In your case the plugin is pytest-asyncio, not a conftest.

The collector.obj causes the module to be imported and the Skipped exception to be raised. But pytest only handles Skipped exceptions from collect(), not earlier.

I think this is something that can be improved in pytest, but it seems that a workaround was already added to pytest-asyncio: https://github.com/pytest-dev/pytest-asyncio/pull/702. So if you update pytest-asyncio the problem should be fixed.