search

jupyter / terminado

Terminals served by tornado websockets
http://terminado.readthedocs.org/en/latest/
BSD 2-Clause "Simplified" License
365 stars 94 forks source link

0.17.1: pytest is failing in `terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals` unit #211

Open kloczek opened 1 year ago

kloczek commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.17.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.17.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/terminado-0.17.1, configfile: pyproject.toml, testpaths: terminado/tests/ collected 9 items terminado/tests/basic_test.py .......F. [100%] ================================================================================= FAILURES ================================================================================= ____________________________________________________________________ UniqueTermTests.test_max_terminals ____________________________________________________________________ self = @tornado.testing.gen_test @pytest.mark.skipif("linux" not in platform, reason="It only works on Linux") async def test_max_terminals(self): tms = await self.get_term_clients(["/unique"] * MAX_TERMS) pids = await self.get_pids(tms) self.assertEqual(len(set(pids)), MAX_TERMS) # All PIDs unique # MAX_TERMS+1 should fail tm = await self.get_term_client("/unique") msg = await tm.read_msg() self.assertEqual(msg, None) # Connection closed # Close one tms[0].close() msg = await tms[0].read_msg() # Closed self.assertEqual(msg, None) # Should be able to open back up to MAX_TERMS tm = await self.get_term_client("/unique") msg = await tm.read_msg() > self.assertEqual(msg[0], "setup") E TypeError: 'NoneType' object is not subscriptable terminado/tests/basic_test.py:307: TypeError ---------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------- WARNING tornado.general:autoreload.py:128 tornado.autoreload started more than once in the same process ERROR tornado.application:web.py:1798 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:43321', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 942, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/websocket.py", line 56, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/management.py", line 323, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals ERROR tornado.application:web.py:1798 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:43321', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 942, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/websocket.py", line 56, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/management.py", line 323, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals ============================================================================= warnings summary ============================================================================= ../../../../../usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1294 /usr/lib/python3.8/site-packages/_pytest/config/__init__.py:1294: PytestConfigWarning: Unknown config option: timeout self._warn_or_fail_if_strict(f"Unknown config option: {key}\n") terminado/tests/basic_test.py:310 /home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/tests/basic_test.py:310: PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html @pytest.mark.timeout(timeout=ASYNC_TEST_TIMEOUT, method="thread") -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================================================================== slowest 10 durations =========================================================================== 8.42s call terminado/tests/basic_test.py::NamedTermTests::test_namespace 6.90s call terminado/tests/basic_test.py::CommonTests::test_basic_command 6.54s call terminado/tests/basic_test.py::NamedTermTests::test_max_terminals 6.54s call terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals 4.39s call terminado/tests/basic_test.py::UniqueTermTests::test_unique_processes 4.30s call terminado/tests/basic_test.py::SingleTermTests::test_single_process 2.59s call terminado/tests/basic_test.py::UniqueTermTests::test_large_io_doesnt_hang 0.80s call terminado/tests/basic_test.py::CommonTests::test_basic 0.13s call terminado/tests/basic_test.py::NamedTermTests::test_new (1 durations < 0.005s hidden. Use -vv to show these durations.) ========================================================================= short test summary info ========================================================================== FAILED terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals - TypeError: 'NoneType' object is not subscriptable ================================================================= 1 failed, 8 passed, 2 warnings in 40.89s ================================================================= ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.12 appdirs 1.4.4 attrs 22.1.0 Babel 2.11.0 Brlapi 0.8.3 build 0.9.0 charset-normalizer 3.0.1 cssselect 1.1.0 distro 1.8.0 dnspython 2.2.1 docutils 0.19 editables 0.3 exceptiongroup 1.0.0 extras 1.0.0 fixtures 4.0.0 gpg 1.17.1-unknown hatchling 1.11.1 idna 3.4 imagesize 1.4.1 importlib-metadata 5.1.0 iniconfig 1.1.1 Jinja2 3.1.2 libcomps 0.1.19 louis 3.24.0 lxml 4.9.1 markdown-it-py 2.1.0 MarkupSafe 2.1.1 mdit-py-plugins 0.3.3 mdurl 0.1.2 myst-parser 0.18.1 numpy 1.23.1 packaging 21.3 pathspec 0.10.2 pbr 5.9.0 pep517 0.13.0 pip 22.3.1 pluggy 1.0.0 ptyprocess 0.7.0 Pygments 2.13.0 PyGObject 3.42.2 pyparsing 3.0.9 pytest 7.2.0 python-dateutil 2.8.2 pytz 2022.4 PyYAML 6.0 requests 2.28.1 rpm 4.17.0 scour 0.38.2 six 1.16.0 snowballstemmer 2.2.0 Sphinx 5.3.0 sphinxcontrib-applehelp 1.0.2.dev20221204 sphinxcontrib-devhelp 1.0.2.dev20221204 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1.dev20221204 sphinxcontrib-qthelp 1.0.3.dev20221204 sphinxcontrib-serializinghtml 1.1.5 testtools 2.5.0 tomli 2.0.1 tornado 6.2 typing_extensions 4.4.0 urllib3 1.26.12 wheel 0.38.4 zipp 3.11.0 ```
kloczek commented 1 year ago

Updated output after install missing pytest-timeout

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.17.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.17.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/terminado-0.17.1, configfile: pyproject.toml, testpaths: terminado/tests/ plugins: timeout-2.1.0 timeout: 300.0s timeout method: signal timeout func_only: False collected 9 items terminado/tests/basic_test.py .......F. [100%] ================================================================================= FAILURES ================================================================================= ____________________________________________________________________ UniqueTermTests.test_max_terminals ____________________________________________________________________ self = @tornado.testing.gen_test @pytest.mark.skipif("linux" not in platform, reason="It only works on Linux") async def test_max_terminals(self): tms = await self.get_term_clients(["/unique"] * MAX_TERMS) pids = await self.get_pids(tms) self.assertEqual(len(set(pids)), MAX_TERMS) # All PIDs unique # MAX_TERMS+1 should fail tm = await self.get_term_client("/unique") msg = await tm.read_msg() self.assertEqual(msg, None) # Connection closed # Close one tms[0].close() msg = await tms[0].read_msg() # Closed self.assertEqual(msg, None) # Should be able to open back up to MAX_TERMS tm = await self.get_term_client("/unique") msg = await tm.read_msg() > self.assertEqual(msg[0], "setup") E TypeError: 'NoneType' object is not subscriptable terminado/tests/basic_test.py:307: TypeError ---------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------- WARNING tornado.general:autoreload.py:128 tornado.autoreload started more than once in the same process ERROR tornado.application:web.py:1798 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:36907', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 942, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/websocket.py", line 56, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/management.py", line 323, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals ERROR tornado.application:web.py:1798 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:36907', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 942, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/websocket.py", line 56, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.17.1/terminado/management.py", line 323, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals =========================================================================== slowest 10 durations =========================================================================== 8.42s call terminado/tests/basic_test.py::NamedTermTests::test_namespace 6.85s call terminado/tests/basic_test.py::CommonTests::test_basic_command 6.61s call terminado/tests/basic_test.py::NamedTermTests::test_max_terminals 6.60s call terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals 4.44s call terminado/tests/basic_test.py::UniqueTermTests::test_unique_processes 4.30s call terminado/tests/basic_test.py::SingleTermTests::test_single_process 2.53s call terminado/tests/basic_test.py::UniqueTermTests::test_large_io_doesnt_hang 0.80s call terminado/tests/basic_test.py::CommonTests::test_basic 0.13s call terminado/tests/basic_test.py::NamedTermTests::test_new (1 durations < 0.005s hidden. Use -vv to show these durations.) ========================================================================= short test summary info ========================================================================== FAILED terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals - TypeError: 'NoneType' object is not subscriptable ======================================================================= 1 failed, 8 passed in 40.95s ======================================================================= ```
kloczek commented 1 year ago

I found yet another small issue. Looks like terminado/tests is installed. It would be good to move that directory to tests/.

opoplawski commented 1 year ago

We're starting to see this in Fedora rawhide: https://koschei.fedoraproject.org/package/python-terminado?collection=f38

There are quite a few dep changes, so hard to say what exactly has triggered it.

opoplawski commented 1 year ago

Seems like it may be intermittent as well.

kloczek commented 9 months ago

Just tested 0.18.0 and now pytest fails in other unit

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.18.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.18.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ============================= test session starts ============================== platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/terminado-0.18.0 configfile: pyproject.toml testpaths: tests/ plugins: timeout-2.2.0 timeout: 300.0s timeout method: signal timeout func_only: False collected 9 items tests/basic_test.py .......F. [100%] =================================== FAILURES =================================== ______________________ UniqueTermTests.test_max_terminals ______________________ self = @tornado.testing.gen_test @pytest.mark.skipif("linux" not in platform, reason="It only works on Linux") async def test_max_terminals(self): tms = await self.get_term_clients(["/unique"] * MAX_TERMS) pids = await self.get_pids(tms) self.assertEqual(len(set(pids)), MAX_TERMS) # All PIDs unique # MAX_TERMS+1 should fail tm = await self.get_term_client("/unique") msg = await tm.read_msg() self.assertEqual(msg, None) # Connection closed # Close one tms[0].close() msg = await tms[0].read_msg() # Closed self.assertEqual(msg, None) # Should be able to open back up to MAX_TERMS tm = await self.get_term_client("/unique") msg = await tm.read_msg() > self.assertEqual(msg[0], "setup") E TypeError: 'NoneType' object is not subscriptable msg = None pids = [662359, 662363, 662372] self = tm = tms = [, , ] tests/basic_test.py:305: TypeError ------------------------------ Captured log call ------------------------------- WARNING tornado.general:autoreload.py:128 tornado.autoreload started more than once in the same process INFO tornado.access:web.py:2344 101 GET /unique (127.0.0.1) 2.49ms INFO terminado.websocket:websocket.py:60 TermSocket.open: None INFO terminado.websocket:websocket.py:67 TermSocket.open: Opened tty INFO tornado.access:web.py:2344 101 GET /unique (127.0.0.1) 16.68ms INFO terminado.websocket:websocket.py:60 TermSocket.open: None INFO terminado.websocket:websocket.py:67 TermSocket.open: Opened tty INFO tornado.access:web.py:2344 101 GET /unique (127.0.0.1) 30.87ms INFO terminado.websocket:websocket.py:60 TermSocket.open: None INFO terminado.websocket:websocket.py:67 TermSocket.open: Opened tty INFO tornado.access:web.py:2344 101 GET /unique (127.0.0.1) 1.69ms INFO terminado.websocket:websocket.py:60 TermSocket.open: None ERROR tornado.application:web.py:1871 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:38509', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 937, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.18.0/terminado/websocket.py", line 64, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.18.0/terminado/management.py", line 347, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals INFO terminado.websocket:websocket.py:120 Websocket closed INFO terminado.management:management.py:355 Websocket closed, sending SIGHUP to terminal. INFO tornado.access:web.py:2344 101 GET /unique (127.0.0.1) 1.04ms INFO terminado.websocket:websocket.py:60 TermSocket.open: None ERROR tornado.application:web.py:1871 Uncaught exception GET /unique (127.0.0.1) HTTPServerRequest(protocol='http', host='127.0.0.1:38509', method='GET', uri='/unique', version='HTTP/1.1', remote_ip='127.0.0.1') Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/tornado/websocket.py", line 937, in _accept_connection open_result = handler.open(*handler.open_args, **handler.open_kwargs) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.18.0/terminado/websocket.py", line 64, in open self.terminal = self.term_manager.get_terminal(url_component) File "/home/tkloczko/rpmbuild/BUILD/terminado-0.18.0/terminado/management.py", line 347, in get_terminal raise MaxTerminalsReached(self.max_terminals) terminado.management.MaxTerminalsReached: Cannot create more than 3 terminals INFO terminado.management:management.py:246 EOF on FD 23; stopping reading INFO terminado.management:management.py:246 EOF on FD 24; stopping reading INFO terminado.management:management.py:246 EOF on FD 25; stopping reading INFO terminado.websocket:websocket.py:120 Websocket closed INFO terminado.management:management.py:355 Websocket closed, sending SIGHUP to terminal. INFO terminado.websocket:websocket.py:120 Websocket closed INFO terminado.management:management.py:355 Websocket closed, sending SIGHUP to terminal. ============================= slowest 10 durations ============================= 8.33s call tests/basic_test.py::NamedTermTests::test_namespace 6.51s call tests/basic_test.py::CommonTests::test_basic_command 6.43s call tests/basic_test.py::UniqueTermTests::test_max_terminals 6.43s call tests/basic_test.py::NamedTermTests::test_max_terminals 4.29s call tests/basic_test.py::UniqueTermTests::test_unique_processes 4.20s call tests/basic_test.py::SingleTermTests::test_single_process 2.40s call tests/basic_test.py::UniqueTermTests::test_large_io_doesnt_hang 0.47s call tests/basic_test.py::CommonTests::test_basic 0.13s call tests/basic_test.py::NamedTermTests::test_new (1 durations < 0.005s hidden. Use -vv to show these durations.) =========================== short test summary info ============================ FAILED tests/basic_test.py::UniqueTermTests::test_max_terminals - TypeError: 'NoneType' object is not subscriptable ========================= 1 failed, 8 passed in 39.56s ========================= ```
opoplawski commented 9 months ago

FWIW - I've not yet seen this with 0.18.0 on Fedora Rawhide(40).

kloczek commented 2 months ago

Just tested 0.18.1 with pytest 8.2.1 and I see new fails

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.18.1-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-terminado-0.18.1-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' --deselect tests/basic_test.py::UniqueTermTests::test_max_terminals ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/terminado-0.18.1 configfile: pyproject.toml testpaths: tests/ plugins: timeout-2.3.1 timeout: 300.0s timeout method: signal timeout func_only: False collected 0 items / 5 errors ==================================== ERRORS ==================================== _____________________ ERROR collecting tests/basic_test.py _____________________ /usr/lib64/python3.10/site-packages/tornado/testing.py:180: in __init__ setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName))) E AttributeError: 'TermTestCase' object has no attribute 'runTest'. Did you mean: 'subTest'? __class__ = methodName = 'runTest' self = _____________________ ERROR collecting tests/basic_test.py _____________________ /usr/lib64/python3.10/site-packages/tornado/testing.py:180: in __init__ setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName))) E AttributeError: 'CommonTests' object has no attribute 'runTest'. Did you mean: 'subTest'? __class__ = methodName = 'runTest' self = _____________________ ERROR collecting tests/basic_test.py _____________________ /usr/lib64/python3.10/site-packages/tornado/testing.py:180: in __init__ setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName))) E AttributeError: 'NamedTermTests' object has no attribute 'runTest'. Did you mean: 'subTest'? __class__ = methodName = 'runTest' self = _____________________ ERROR collecting tests/basic_test.py _____________________ /usr/lib64/python3.10/site-packages/tornado/testing.py:180: in __init__ setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName))) E AttributeError: 'SingleTermTests' object has no attribute 'runTest'. Did you mean: 'subTest'? __class__ = methodName = 'runTest' self = _____________________ ERROR collecting tests/basic_test.py _____________________ /usr/lib64/python3.10/site-packages/tornado/testing.py:180: in __init__ setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName))) E AttributeError: 'UniqueTermTests' object has no attribute 'runTest'. Did you mean: 'subTest'? __class__ = methodName = 'runTest' self = =========================== short test summary info ============================ ERROR tests/basic_test.py::TermTestCase - AttributeError: 'TermTestCase' object has no attribute 'runTest'. Did you m... ERROR tests/basic_test.py::CommonTests - AttributeError: 'CommonTests' object has no attribute 'runTest'. Did you me... ERROR tests/basic_test.py::NamedTermTests - AttributeError: 'NamedTermTests' object has no attribute 'runTest'. Did you... ERROR tests/basic_test.py::SingleTermTests - AttributeError: 'SingleTermTests' object has no attribute 'runTest'. Did yo... ERROR tests/basic_test.py::UniqueTermTests - AttributeError: 'UniqueTermTests' object has no attribute 'runTest'. Did yo... !!!!!!!!!!!!!!!!!!! Interrupted: 5 errors during collection !!!!!!!!!!!!!!!!!!!! ============================== 5 errors in 0.32s =============================== ```