search

pytest-dev / pytest-twisted

test twisted code with pytest
BSD 3-Clause "New" or "Revised" License
46 stars 24 forks source link

1.14.1: pytest fails in `test_async_fixture_module_scope.py::test_second` unit #176

Open kloczek opened 5 months ago

kloczek commented 5 months ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-twisted-1.14.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-twisted-1.14.1-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' -q ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1 configfile: pytest.ini plugins: twisted-1.14.1, hypothesis-6.100.0 collected 60 items testing/test_basic.py .....................................ss.ssF....... [ 83%] .......... [100%] =================================== FAILURES =================================== _______________________ test_async_fixture_module_scope ________________________ testdir = cmd_opts = ('/usr/bin/python3', '-m', 'pytest', '-v', '--reactor=default') @skip_if_no_async_generators() def test_async_fixture_module_scope(testdir, cmd_opts): test_file = """ from twisted.internet import reactor, defer import pytest import pytest_twisted check_me = 0 @pytest_twisted.async_yield_fixture(scope="module") async def foo(): global check_me if check_me != 0: raise Exception('check_me already modified before fixture run') check_me = 1 yield 42 if check_me != 3: raise Exception( 'check_me not updated properly: {}'.format(check_me), ) check_me = 0 def test_first(foo): global check_me assert check_me == 1 assert foo == 42 check_me = 2 def test_second(foo): global check_me assert check_me == 2 assert foo == 42 check_me = 3 """ testdir.makepyfile(test_file) rr = testdir.run(*cmd_opts, timeout=timeout) > assert_outcomes(rr, {"passed": 2}) /home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1/testing/test_basic.py:1055: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ run_result = outcomes = {'passed': 2} def assert_outcomes(run_result, outcomes): formatted_output = format_run_result_output_for_assert(run_result) try: result_outcomes = run_result.parseoutcomes() except ValueError: assert False, formatted_output normalized_result_outcomes = { force_plural(name): outcome for name, outcome in result_outcomes.items() if name != "seconds" } > assert normalized_result_outcomes == outcomes, formatted_output E AssertionError: E ---- stdout E ============================= test session starts ============================== E platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 -- /usr/bin/python3 E cachedir: .pytest_cache E hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0/.hypothesis/examples')) E rootdir: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0 E plugins: twisted-1.14.1, hypothesis-6.100.0 E collecting ... collected 2 items E E test_async_fixture_module_scope.py::test_first PASSED [ 50%] E test_async_fixture_module_scope.py::test_second ERROR [100%] E E ==================================== ERRORS ==================================== E ________________________ ERROR at setup of test_second _________________________ E E @pytest_twisted.async_yield_fixture(scope="module") E async def foo(): E global check_me E E if check_me != 0: E raise Exception('check_me already modified before fixture run') E E check_me = 1 E E yield 42 E E if check_me != 3: E > raise Exception( E 'check_me not updated properly: {}'.format(check_me), E ) E E Exception: check_me not updated properly: 2 E E test_async_fixture_module_scope.py:19: Exception E =========================== short test summary info ============================ E ERROR test_async_fixture_module_scope.py::test_second - Exception: check_me n... E ========================== 1 passed, 1 error in 0.74s ========================== E ---- stderr E E ---- E E assert {'errors': 1, 'passed': 1} == {'passed': 2} E E Differing items: E {'passed': 1} != {'passed': 2} E Left contains 1 more item: E {'errors': 1} E Use -v to get more diff /home/tkloczko/rpmbuild/BUILD/pytest-twisted-1.14.1/testing/test_basic.py:45: AssertionError ----------------------------- Captured stdout call ----------------------------- running: /usr/bin/python3 -m pytest -v --reactor=default in: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0 ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0/.hypothesis/examples')) rootdir: /tmp/pytest-of-tkloczko/pytest-33/test_async_fixture_module_scope0 plugins: twisted-1.14.1, hypothesis-6.100.0 collecting ... collected 2 items test_async_fixture_module_scope.py::test_first PASSED [ 50%] test_async_fixture_module_scope.py::test_second ERROR [100%] ==================================== ERRORS ==================================== ________________________ ERROR at setup of test_second _________________________ @pytest_twisted.async_yield_fixture(scope="module") async def foo(): global check_me if check_me != 0: raise Exception('check_me already modified before fixture run') check_me = 1 yield 42 if check_me != 3: > raise Exception( 'check_me not updated properly: {}'.format(check_me), ) E Exception: check_me not updated properly: 2 test_async_fixture_module_scope.py:19: Exception =========================== short test summary info ============================ ERROR test_async_fixture_module_scope.py::test_second - Exception: check_me n... ========================== 1 passed, 1 error in 0.74s ========================== =========================== short test summary info ============================ SKIPPED [2] testing/test_basic.py:77: reactor is default not qt5reactor SKIPPED [2] testing/test_basic.py:77: reactor is default not asyncio FAILED testing/test_basic.py::test_async_fixture_module_scope - AssertionError: ============== 1 failed, 55 passed, 4 skipped in 79.06s (0:01:19) ============== /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_get_no_perms0 : [Errno 39] Directory not empty: 'test_safe_get_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_set_no_perms0 : [Errno 39] Directory not empty: 'test_safe_set_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_safe_delete_no_perms0 : [Errno 39] Directory not empty: 'test_safe_delete_no_perms0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826/test_rmtree_errorhandler_rerai0 : [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0' warnings.warn( /usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826 : [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-aca48b6b-259c-4520-8811-f9d7aedc6826' warnings.warn( ```
List of installed modules in build env: ```console Package Version ------------------ ----------- attrs 23.2.0 Automat 22.10.0 build 1.2.1 constantly 23.10.4 decorator 5.1.1 exceptiongroup 1.1.3 greenlet 3.0.3 hyperlink 21.0.0 hypothesis 6.100.0 idna 3.7 importlib_metadata 7.1.0 incremental 22.10.0 iniconfig 2.0.0 installer 0.7.0 packaging 24.0 pluggy 1.5.0 pyproject_hooks 1.0.0 pytest 8.2.1 python-dateutil 2.9.0.post0 setuptools 69.4.0 sortedcontainers 2.4.0 tokenize_rt 5.2.0 tomli 2.0.1 Twisted 24.3.0 typing_extensions 4.12.1 wheel 0.43.0 zipp 3.19.1 zope.event 5.0 zope.interface 6.4.post2 ```

Please let me know if you need more details or want me to perform some diagnostics.

mgorny commented 4 months ago

This is caused by pytest-8.2, and can be reproduced using tox.

@kloczek, could you update the summary to point the cause?

altendky commented 4 months ago

Looks like this is present in the last (awhile ago) scheduled CI run. https://github.com/pytest-dev/pytest-twisted/actions/runs/9460305008/job/26058879367#step:10:252 I'll see what I can do. Thanks for pointing this out.

RonnyPfannschmidt commented 4 months ago

https://github.com/pytest-dev/pytest-twisted/blob/ba5eb8892647b7b0834151f021318bc650cf129c/pytest_twisted.py#L336 needs to use the cache key

altendky commented 4 months ago

https://pypi.org/project/pytest-twisted/1.14.2/

v1.14.2 is available. @kloczek, let me know if that works for you or not. I'm sorry for leaving CI broken and making you find this issue instead.

mgorny commented 4 months ago

If that's worth anything, it works for me now. Thanks!