Bad and misleading test

s0undt3ch commented 9 months ago

I believe there's a test that is not testing anything anymore. The test was introduced in https://github.com/pytest-dev/pytest-timeout/pull/88, https://github.com/pytest-dev/pytest-timeout/commit/1a35f94298c1260500907bd674b75cf59df0c949

================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.2, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/pytest-of-vampas/pytest-4/test_not_main_thread0
plugins: github-actions-annotate-failures-0.2.0, timeout-2.2.0, cov-4.1.0
timeout: 1.0s
timeout method: signal
timeout func_only: False
collected 0 items / 1 error
====================================================================================================== ERRORS =======================================================================================================
_____________________________________________________________________________________ ERROR collecting test_not_main_thread.py ______________________________________________________________________________________
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/python.py:531: in collect
    self._inject_setup_module_fixture()
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/python.py:545: in _inject_setup_module_fixture
    self.obj, ("setUpModule", "setup_module")
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/python.py:310: in obj
    self._obj = obj = self._getobj()
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/python.py:528: in _getobj
    return self._importtestmodule()
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/python.py:617: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/pathlib.py:567: in import_path
    importlib.import_module(module_name)
/home/vampas/.pyenv/versions/3.10.2/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
/home/vampas/projects/SaltStack/pytest/timeout/hotfix/settings-hook/.tox/py310/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
    exec(co, module.__dict__)
test_not_main_thread.py:4: in <module>
    current_timeout_setup = pytest_timeout.timeout_setup
E   AttributeError: module 'pytest_timeout' has no attribute 'timeout_setup'
============================================================================================== short test summary info ==============================================================================================
ERROR test_not_main_thread.py - AttributeError: module 'pytest_timeout' has no attribute 'timeout_setup'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================================= 1 error in 0.20s ==================================================================================================

Is this really what is getting tested? The error?

s0undt3ch commented 9 months ago

And then, if the purpose was to see if signal would turn into thread, the test doesn't check for that?!

flub commented 9 months ago

Yes, that seems like something started going wrong with that test. Whether it was always broken or broke over time and silently succeeded due to the separate reporting of the timeout in the header I don't know, but doesn't really matter.

@s0undt3ch Could you add a PR fixing that test? Thanks!

pytest-dev / pytest-timeout

Bad and misleading test #158