search

zeromq / pyzmq

PyZMQ: Python bindings for zeromq
http://zguide.zeromq.org/py:all
BSD 3-Clause "New" or "Revised" License
3.66k stars 638 forks source link

[TST]: pytest unable to find `zmq_test_utils` module with --import-mode=importlib #2020

Closed kloczek closed 3 weeks ago

kloczek commented 3 weeks ago

This is a pyzmq bug

What pyzmq version?

26.1.1

What libzmq version?

4.3.5

Python version (and how it was installed)

3.10.14

OS

Linux

What happened?

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Looks like ATM pytest fails because missing zmq_test_utils module.

Code to reproduce bug

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-zmq-26.1.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-zmq-26.1.1-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' --import-mode=importlib ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1 configfile: pytest.ini testpaths: tests plugins: asyncio-0.23.8 asyncio: mode=auto collected 114 items / 22 errors / 1 skipped ========================================================================================== ERRORS =========================================================================================== ____________________________________________________________________________ ERROR collecting tests/test_auth.py ____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_auth.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_auth.py:17: in from zmq_test_utils import SkipTest, skip_pypy E ModuleNotFoundError: No module named 'zmq_test_utils' ________________________________________________________________________ ERROR collecting tests/test_cffi_backend.py ________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_cffi_backend.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_cffi_backend.py:4: in from zmq_test_utils import SkipTest E ModuleNotFoundError: No module named 'zmq_test_utils' __________________________________________________________________________ ERROR collecting tests/test_context.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_context.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_context.py:17: in from zmq_test_utils import PYPY, BaseZMQTestCase, GreenTest, SkipTest E ModuleNotFoundError: No module named 'zmq_test_utils' _________________________________________________________________________ ERROR collecting tests/test_decorators.py _________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_decorators.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_decorators.py:7: in from zmq_test_utils import BaseZMQTestCase, term_context E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_device.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_device.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_device.py:8: in from zmq_test_utils import PYPY, BaseZMQTestCase, GreenTest, SkipTest, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_draft.py ____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_draft.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_draft.py:9: in from zmq_test_utils import BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_error.py ____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_error.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_error.py:8: in from zmq_test_utils import BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_future.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_future.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_future.py:17: in from zmq_test_utils import BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' ____________________________________________________________________________ ERROR collecting tests/test_log.py _____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_log.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_log.py:10: in from zmq_test_utils import BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' __________________________________________________________________________ ERROR collecting tests/test_message.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_message.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_message.py:19: in from zmq_test_utils import PYPY, BaseZMQTestCase, SkipTest, skip_pypy E ModuleNotFoundError: No module named 'zmq_test_utils' __________________________________________________________________________ ERROR collecting tests/test_monitor.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_monitor.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_monitor.py:7: in from zmq_test_utils import require_zmq_4 E ModuleNotFoundError: No module named 'zmq_test_utils' __________________________________________________________________________ ERROR collecting tests/test_monqueue.py __________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_monqueue.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_monqueue.py:9: in from zmq_test_utils import PYPY, BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' _________________________________________________________________________ ERROR collecting tests/test_multipart.py __________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_multipart.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_multipart.py:6: in from zmq_test_utils import BaseZMQTestCase, GreenTest, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ____________________________________________________________________________ ERROR collecting tests/test_pair.py ____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_pair.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_pair.py:6: in from zmq_test_utils import BaseZMQTestCase, GreenTest, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ____________________________________________________________________________ ERROR collecting tests/test_poll.py ____________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_poll.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_poll.py:11: in from zmq_test_utils import GreenTest, PollZMQTestCase, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ______________________________________________________________________ ERROR collecting tests/test_proxy_steerable.py _______________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_proxy_steerable.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_proxy_steerable.py:9: in from zmq_test_utils import PYPY, BaseZMQTestCase, SkipTest E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_pubsub.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_pubsub.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_pubsub.py:8: in from zmq_test_utils import BaseZMQTestCase, GreenTest, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_reqrep.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_reqrep.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_reqrep.py:6: in from zmq_test_utils import BaseZMQTestCase, GreenTest, have_gevent E ModuleNotFoundError: No module named 'zmq_test_utils' ________________________________________________________________________ ERROR collecting tests/test_retry_eintr.py _________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_retry_eintr.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_retry_eintr.py:11: in from zmq_test_utils import BaseZMQTestCase, SkipTest E ModuleNotFoundError: No module named 'zmq_test_utils' __________________________________________________________________________ ERROR collecting tests/test_security.py __________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_security.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_security.py:13: in from zmq_test_utils import PYPY, BaseZMQTestCase, SkipTest E ModuleNotFoundError: No module named 'zmq_test_utils' ___________________________________________________________________________ ERROR collecting tests/test_socket.py ___________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_socket.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_socket.py:19: in from zmq_test_utils import BaseZMQTestCase, GreenTest, SkipTest, have_gevent, skip_pypy E ModuleNotFoundError: No module named 'zmq_test_utils' _________________________________________________________________________ ERROR collecting tests/test_win32_shim.py _________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/pyzmq-26.1.1/tests/test_win32_shim.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_win32_shim.py:8: in from zmq_test_utils import BaseZMQTestCase E ModuleNotFoundError: No module named 'zmq_test_utils' ===================================================================================== warnings summary ====================================================================================== ../../../../../usr/lib/python3.10/site-packages/pytest_asyncio/plugin.py:223 /usr/lib/python3.10/site-packages/pytest_asyncio/plugin.py:223: PytestDeprecationWarning: io_loop is asynchronous and explicitly requests the "event_loop" fixture. Asynchronous fixtures and test functions should use "asyncio.get_running_loop()" instead. warnings.warn( -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_mypy.py:12: could not import 'mypy': No module named 'mypy' ERROR tests/test_auth.py ERROR tests/test_cffi_backend.py ERROR tests/test_context.py ERROR tests/test_decorators.py ERROR tests/test_device.py ERROR tests/test_draft.py ERROR tests/test_error.py ERROR tests/test_future.py ERROR tests/test_log.py ERROR tests/test_message.py ERROR tests/test_monitor.py ERROR tests/test_monqueue.py ERROR tests/test_multipart.py ERROR tests/test_pair.py ERROR tests/test_poll.py ERROR tests/test_proxy_steerable.py ERROR tests/test_pubsub.py ERROR tests/test_reqrep.py ERROR tests/test_retry_eintr.py ERROR tests/test_security.py ERROR tests/test_socket.py ERROR tests/test_win32_shim.py !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 22 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ========================================================================= 1 skipped, 1 warning, 22 errors in 0.95s ========================================================================== ```

More info

List of installed modules in build env: ```console Package Version ----------------------------- ----------- alabaster 0.7.16 apeye 1.4.1 apeye-core 1.1.4 autodocsumm 0.2.13 Babel 2.15.0 beautifulsoup4 4.12.3 build 1.2.1 CacheControl 0.14.0 cffi 1.16.0 charset-normalizer 3.3.2 cssutils 2.11.1 Cython 3.0.11 defusedxml 0.7.1 dict2css 0.3.0.post1 distro 1.9.0 docutils 0.20.1 domdf_python_tools 3.8.0.post2 enum-tools 0.12.0 exceptiongroup 1.1.3 filelock 3.15.4 gevent 24.2.1 greenlet 3.0.3 html5lib 1.1 imagesize 1.4.1 importlib_metadata 8.0.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.4 linkify-it-py 2.0.3 markdown-it-py 3.0.0 MarkupSafe 2.1.5 mdit-py-plugins 0.4.1 mdurl 0.1.2 more-itertools 10.3.0 msgpack 1.0.8 myst-parser 4.0.0 natsort 8.4.0 packaging 24.0 pathspec 0.12.1 platformdirs 4.2.2 pluggy 1.5.0 ply 3.11 pycparser 2.22 Pygments 2.18.0 pyproject_hooks 1.0.0 pyproject-metadata 0.8.0 pytest 8.2.2 pytest-asyncio 0.23.8 python-dateutil 2.9.0.post0 PyYAML 6.0.2 requests 2.32.3 ruamel.yaml 0.18.5 ruamel.yaml.clib 0.2.8 scikit_build_core 0.9.8 setuptools 72.2.0 setuptools-scm 8.1.0 snowballstemmer 2.2.0 soupsieve 2.6 Sphinx 8.0.2 sphinx-autodoc-typehints 2.2.3 sphinx-jinja2-compat 0.3.0 sphinx-prompt 1.6.0 sphinx_rtd_theme 2.0.1a1 sphinx-tabs 3.4.5 sphinx-toolbox 3.8.0 sphinxcontrib-applehelp 1.0.8 sphinxcontrib-devhelp 1.0.6 sphinxcontrib-htmlhelp 2.0.5 sphinxcontrib-jquery 4.1 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.7 sphinxcontrib-serializinghtml 2.0.0 tabulate 0.9.0 tokenize_rt 6.0.0 tomli 2.0.1 tornado 6.4.1 typing_extensions 4.12.2 uc-micro-py 1.0.3 urllib3 2.2.2 webencodings 0.5.1 wheel 0.44.0 zipp 3.19.2 zope.event 5.0 zope.interface 7.0.1 ```
kloczek commented 3 weeks ago

BTW it is yest another issue related to use "test as installed" testing. zmq is using relative imports which forces to have exact relative location of the zmq I have patch which removes use of those relative imports. Do you want to me to submit that patch as PR to review an/or merge? 🤔

minrk commented 3 weeks ago

I don't believe this is related to relative imports, since the file is not part of the pyzmq package and unrelated to installation and also not a relative import. It is part of the test suite, which pytest puts on sys.path by default. You've opted out of pytest's standard behavior with --import-mode=importlib, which is what prevents it from working. Removing that argument to leave pytest to its own standard behavior is the simplest fix.

I can make a patch to allow importlib to work as well.