tomerfiliba-org / rpyc

RPyC (Remote Python Call) - A transparent and symmetric RPC library for python
http://rpyc.readthedocs.org
Other
1.58k stars 245 forks source link

5.3.1: pytest is failing in `tests/test_win32pipes.py::Test_NamedPipe::test_rpyc` unit #544

Open kloczek opened 1 year ago

kloczek commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.18, pytest-7.4.2, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-5.3.1 collected 88 items tests/test_affinity.py . [ 1%] tests/test_async.py ..... [ 6%] tests/test_attr_access.py ......... [ 17%] tests/test_attributes.py . [ 18%] tests/test_brine.py . [ 19%] tests/test_classic.py ..... [ 25%] tests/test_context_managers.py .. [ 27%] tests/test_custom_service.py ....... [ 35%] tests/test_dataclass.py . [ 36%] tests/test_deploy.py ...s [ 40%] tests/test_gdb.py . [ 42%] tests/test_get_id_pack.py .... [ 46%] tests/test_gevent_server.py sss [ 50%] tests/test_ipv6.py s [ 51%] tests/test_magic.py .. [ 53%] tests/test_netref_hierachy.py ...... [ 60%] tests/test_oneshot_server.py . [ 61%] tests/test_refcount.py . [ 62%] tests/test_registry.py ...... [ 69%] tests/test_remote_exception.py .. [ 71%] tests/test_remoting.py s.s.s [ 77%] tests/test_rpyc_over_rpyc.py ... [ 80%] tests/test_service_pickle.py . [ 81%] tests/test_ssh.py .. [ 84%] tests/test_ssl.py . [ 85%] tests/test_teleportation.py ...... [ 92%] tests/test_threaded_server.py ... [ 95%] tests/test_threads.py . [ 96%] tests/test_win32pipes.py ssF [100%] ========================================================================================= FAILURES ========================================================================================== _________________________________________________________________________________ Test_NamedPipe.test_rpyc __________________________________________________________________________________ self = def test_rpyc(self): > assert self.client.root.get_service_name() == "VOID" E AttributeError: 'Test_NamedPipe' object has no attribute 'client' tests/test_win32pipes.py:57: AttributeError ===================================================================================== warnings summary ====================================================================================== tests/test_ssh.py::Test_Ssh::test_connect /usr/lib/python3.8/site-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread RpycSpawnThread-builtins.method-139822925896352-139816296592192 Traceback (most recent call last): File "/usr/lib64/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/lib64/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/netref.py", line 240, in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/netref.py", line 63, in syncreq return conn.sync_request(handler, proxy, *args) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 718, in sync_request return _async_res.value File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/async_.py", line 106, in value self.wait() File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/async_.py", line 51, in wait self._conn.serve(self._ttl) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 447, in serve self._dispatch(data) # Dispatch will unbox, invoke callbacks, etc. File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 399, in _dispatch self._dispatch_request(seq, args) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 373, in _dispatch_request self._send(consts.MSG_REPLY, seq, self._box(res)) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 298, in _send self._channel.send(data) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/channel.py", line 82, in send self.stream.write(header + data[:part1]) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/stream.py", line 288, in write count = self.sock.send(data[:self.MAX_IO_CHUNK]) File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/stream.py", line 96, in __getattr__ raise EOFError("stream has been closed") EOFError: stream has been closed warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg)) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_deploy.py:82: Paramiko is not available SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available SKIPPED [1] tests/test_ipv6.py:19: requires IPv6 SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm SKIPPED [1] tests/test_win32pipes.py:11: Requires windows SKIPPED [1] tests/test_win32pipes.py:24: Requires windows FAILED tests/test_win32pipes.py::Test_NamedPipe::test_rpyc - AttributeError: 'Test_NamedPipe' object has no attribute 'client' ============================================================== 1 failed, 77 passed, 10 skipped, 1 warning in 83.96s (0:01:23) =============================================================== ```

Here is list of installed modules in build env

```console Package Version ----------------------------- -------- alabaster 0.7.13 Babel 2.12.1 build 1.0.3 charset-normalizer 3.2.0 distro 1.8.0 docutils 0.20.1 editables 0.5 exceptiongroup 1.1.3 gpg 1.21.0 hatchling 1.18.0 idna 3.4 imagesize 1.4.1 importlib-metadata 6.8.0 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.2 libcomps 0.1.19 MarkupSafe 2.1.3 numpy 1.24.4 packaging 23.1 pandas 2.0.3 pathspec 0.11.2 pkg 0.0.0 pluggy 1.3.0 plumbum 1.8.2 Pygments 2.16.1 pyproject_hooks 1.0.0 pytest 7.4.2 python-dateutil 2.8.2 pytz 2023.3 requests 2.31.0 six 1.16.0 snowballstemmer 2.2.0 Sphinx 7.0.1 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.3 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.9 tomli 2.0.1 trove-classifiers 2023.8.9 urllib3 1.26.16 wheel 0.41.1 zipp 3.16.2 ```
comrumino commented 1 year ago

I pushed a commit that should resolve this failure. However, I don't have a windows machine currently configured for development. Let me know if that does the trick! Thanks!

kloczek commented 9 months ago

Just tested 6.0.0 and looks like now pytest fails because test suite cannot find support and cfg_tests modules

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.18, pytest-8.0.1, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0 plugins: anyio-4.2.0, nbval-0.10.0 collected 93 items / 2 errors ========================================================================================== ERRORS =========================================================================================== __________________________________________________________________________ ERROR collecting tests/test_affinity.py __________________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_affinity.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib64/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_affinity.py:4: in import support E ModuleNotFoundError: No module named 'support' _______________________________________________________________________ ERROR collecting tests/test_service_pickle.py _______________________________________________________________________ ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_service_pickle.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib64/python3.8/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_service_pickle.py:7: in import cfg_tests E ModuleNotFoundError: No module named 'cfg_tests' ================================================================================== short test summary info ================================================================================== ERROR tests/test_affinity.py ERROR tests/test_service_pickle.py !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ===================================================================================== 2 errors in 0.89s ===================================================================================== ```
kloczek commented 9 months ago

Below quick patch fixed scanning

--- a/tests/test_service_pickle.py
+++ b/tests/test_service_pickle.py
@@ -4,7 +4,7 @@
 import timeit
 import rpyc
 import unittest
-import cfg_tests
+import tests.cfg_tests
 try:
     import pandas as pd
     import numpy as np
--- a/tests/test_affinity.py
+++ b/tests/test_affinity.py
@@ -1,7 +1,7 @@
 import sys
 import time
 import unittest
-import support
+import tests.support
 import rpyc

But pytest started failing in more units

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.18, pytest-8.0.1, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0 plugins: anyio-4.2.0, nbval-0.10.0 collected 95 items tests/test_affinity.py E [ 1%] tests/test_async.py ..... [ 6%] tests/test_attr_access.py ......... [ 15%] tests/test_attributes.py F [ 16%] tests/test_brine.py . [ 17%] tests/test_classic.py ...F. [ 23%] tests/test_context_managers.py .. [ 25%] tests/test_custom_service.py ....... [ 32%] tests/test_dataclass.py . [ 33%] tests/test_deploy.py ...s [ 37%] tests/test_gdb.py . [ 38%] tests/test_get_id_pack.py .... [ 43%] tests/test_gevent_server.py sss [ 46%] tests/test_ipv6.py s [ 47%] tests/test_magic.py FF [ 49%] tests/test_netref_hierachy.py ......F.... [ 61%] tests/test_oneshot_server.py . [ 62%] tests/test_race.py . [ 63%] tests/test_refcount.py . [ 64%] tests/test_registry.py ...... [ 70%] tests/test_remote_exception.py .. [ 72%] tests/test_remoting.py s.s.s [ 77%] tests/test_rpyc_over_rpyc.py ... [ 81%] tests/test_service_pickle.py F [ 82%] tests/test_ssh.py .. [ 84%] tests/test_ssl.py . [ 85%] tests/test_teleportation.py ...... [ 91%] tests/test_threaded_server.py ... [ 94%] tests/test_threads.py . [ 95%] tests/test_urllib3.py s [ 96%] tests/test_win32pipes.py sss [100%] ========================================================================================== ERRORS =========================================================================================== _____________________________________________________________________ ERROR at setup of Test_Affinity.test_pinned_to_0 ______________________________________________________________________ cls = @classmethod def setUpClass(cls): """Construct the a copy of ClassicServer that embeds a sleep(0) into _dispatch and set affinity""" cls._orig_func = rpyc.core.protocol.Connection._dispatch def _sleepy_dispatch(self, data): time.sleep(0.0) return cls._orig_func(self, data) setattr(rpyc.core.protocol.Connection, '_dispatch', _sleepy_dispatch) cls.cfg = {'sync_request_timeout': 5} if sys.platform != "linux": print("Running Test_Affinity is less productive on non-linux systems...") try: cls._skip = None cls._os = None cls._supported = True > cls._os = support.import_module('os', fromlist=('sched_setaffinity', 'sched_getaffinity')) E NameError: name 'support' is not defined tests/test_affinity.py:34: NameError ========================================================================================= FAILURES ========================================================================================== ______________________________________________________________________________ TestAttributes.test_properties _______________________________________________________________________________ self = def test_properties(self): > p = self.conn.modules["test_attributes"].Properties() tests/test_attributes.py:29: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rpyc/core/service.py:132: in __getitem__ self.__cache[name] = self.__getmodule(name) rpyc/core/netref.py:239: in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) rpyc/core/netref.py:63: in syncreq return conn.sync_request(handler, proxy, *args) rpyc/core/protocol.py:744: in sync_request return _async_res.value _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @property def value(self): """Returns the result of the operation. If the result has not yet arrived, accessing this property will wait for it. If the result does not arrive before the expiry time elapses, :class:`AsyncResultTimeout` is raised. If the returned result is an exception, it will be raised here. Otherwise, the result is returned directly. """ self.wait() if self._is_exc: > raise self._obj E _get_exception_class..Derived: No module named 'test_attributes' E E ========= Remote Traceback (1) ========= E Traceback (most recent call last): E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request E res = self._HANDLERS[handler](self, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call E return obj(*args, **dict(kwargs)) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule E return importlib.import_module(name) E File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module E return _bootstrap._gcd_import(name[level:], package, level) E File "", line 1014, in _gcd_import E File "", line 991, in _find_and_load E File "", line 973, in _find_and_load_unlocked E ModuleNotFoundError: No module named 'test_attributes' rpyc/core/async_.py:111: ModuleNotFoundError _________________________________________________________________________________ ClassicMode.test_modules __________________________________________________________________________________ self = def test_modules(self): > self.assertIn('test_magic', self.conn.modules) E AssertionError: 'test_magic' not found in tests/test_classic.py:55: AssertionError ____________________________________________________________________________ TestContextManagers.test_hash_class ____________________________________________________________________________ self = , name = 'test_magic' def __getattr__(self, name): """Provides dot notation access to modules""" try: > return self[name] rpyc/core/service.py:138: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rpyc/core/service.py:132: in __getitem__ self.__cache[name] = self.__getmodule(name) rpyc/core/netref.py:239: in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) rpyc/core/netref.py:63: in syncreq return conn.sync_request(handler, proxy, *args) rpyc/core/protocol.py:744: in sync_request return _async_res.value _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @property def value(self): """Returns the result of the operation. If the result has not yet arrived, accessing this property will wait for it. If the result does not arrive before the expiry time elapses, :class:`AsyncResultTimeout` is raised. If the returned result is an exception, it will be raised here. Otherwise, the result is returned directly. """ self.wait() if self._is_exc: > raise self._obj E _get_exception_class..Derived: No module named 'test_magic' E E ========= Remote Traceback (1) ========= E Traceback (most recent call last): E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request E res = self._HANDLERS[handler](self, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call E return obj(*args, **dict(kwargs)) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule E return importlib.import_module(name) E File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module E return _bootstrap._gcd_import(name[level:], package, level) E File "", line 1014, in _gcd_import E File "", line 991, in _find_and_load E File "", line 973, in _find_and_load_unlocked E ModuleNotFoundError: No module named 'test_magic' rpyc/core/async_.py:111: ModuleNotFoundError During handling of the above exception, another exception occurred: self = def test_hash_class(self): hesh = self.conn.builtins.hash > mod = self.conn.modules.test_magic tests/test_magic.py:40: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , name = 'test_magic' def __getattr__(self, name): """Provides dot notation access to modules""" try: return self[name] except ImportError: > raise AttributeError(name) E AttributeError: test_magic rpyc/core/service.py:140: AttributeError _____________________________________________________________________________ TestContextManagers.test_hash_obj _____________________________________________________________________________ self = , name = 'test_magic' def __getattr__(self, name): """Provides dot notation access to modules""" try: > return self[name] rpyc/core/service.py:138: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rpyc/core/service.py:132: in __getitem__ self.__cache[name] = self.__getmodule(name) rpyc/core/netref.py:239: in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) rpyc/core/netref.py:63: in syncreq return conn.sync_request(handler, proxy, *args) rpyc/core/protocol.py:744: in sync_request return _async_res.value _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @property def value(self): """Returns the result of the operation. If the result has not yet arrived, accessing this property will wait for it. If the result does not arrive before the expiry time elapses, :class:`AsyncResultTimeout` is raised. If the returned result is an exception, it will be raised here. Otherwise, the result is returned directly. """ self.wait() if self._is_exc: > raise self._obj E _get_exception_class..Derived: No module named 'test_magic' E E ========= Remote Traceback (1) ========= E Traceback (most recent call last): E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request E res = self._HANDLERS[handler](self, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call E return obj(*args, **dict(kwargs)) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule E return importlib.import_module(name) E File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module E return _bootstrap._gcd_import(name[level:], package, level) E File "", line 1014, in _gcd_import E File "", line 991, in _find_and_load E File "", line 973, in _find_and_load_unlocked E ModuleNotFoundError: No module named 'test_magic' rpyc/core/async_.py:111: ModuleNotFoundError During handling of the above exception, another exception occurred: self = def test_hash_obj(self): hesh = self.conn.builtins.hash > mod = self.conn.modules.test_magic tests/test_magic.py:56: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , name = 'test_magic' def __getattr__(self, name): """Provides dot notation access to modules""" try: return self[name] except ImportError: > raise AttributeError(name) E AttributeError: test_magic rpyc/core/service.py:140: AttributeError ________________________________________________________________ Test_Netref_Hierarchy.test_instancecheck_across_connections ________________________________________________________________ self = def test_instancecheck_across_connections(self): self.conn2 = rpyc.classic.connect('localhost', port=18878) > self.conn.execute('import test_magic') tests/test_netref_hierachy.py:111: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rpyc/core/netref.py:239: in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) rpyc/core/netref.py:63: in syncreq return conn.sync_request(handler, proxy, *args) rpyc/core/protocol.py:744: in sync_request return _async_res.value _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @property def value(self): """Returns the result of the operation. If the result has not yet arrived, accessing this property will wait for it. If the result does not arrive before the expiry time elapses, :class:`AsyncResultTimeout` is raised. If the returned result is an exception, it will be raised here. Otherwise, the result is returned directly. """ self.wait() if self._is_exc: > raise self._obj E _get_exception_class..Derived: No module named 'test_magic' E E ========= Remote Traceback (1) ========= E Traceback (most recent call last): E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request E res = self._HANDLERS[handler](self, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call E return obj(*args, **dict(kwargs)) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 152, in execute E execute(text, self.namespace) E File "", line 1, in E ModuleNotFoundError: No module named 'test_magic' rpyc/core/async_.py:111: ModuleNotFoundError _________________________________________________________________________ TestServicePickle.test_dataframe_pickling _________________________________________________________________________ self = def setUp(self): self.cfg = {'allow_pickle': True} self.server = rpyc.utils.server.ThreadedServer(MyService, port=0, protocol_config=self.cfg.copy()) self.server.logger.quiet = False self.thd = self.server._start_in_thread() self.conn = rpyc.connect("localhost", self.server.port, config=self.cfg) self.conn2 = rpyc.connect("localhost", self.server.port, config=self.cfg) # globals are made available to timeit, prepare them > cfg_tests.timeit['conn'] = self.conn E NameError: name 'cfg_tests' is not defined tests/test_service_pickle.py:61: NameError ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_deploy.py:86: Paramiko is not available SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available SKIPPED [1] tests/test_ipv6.py:19: requires IPv6 SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available SKIPPED [1] tests/test_win32pipes.py:11: Requires windows SKIPPED [1] tests/test_win32pipes.py:24: Requires windows SKIPPED [1] tests/test_win32pipes.py:56: Requires windows ERROR tests/test_affinity.py::Test_Affinity::test_pinned_to_0 - NameError: name 'support' is not defined FAILED tests/test_attributes.py::TestAttributes::test_properties - _get_exception_class..Derived: No module named 'test_attributes' FAILED tests/test_classic.py::ClassicMode::test_modules - AssertionError: 'test_magic' not found in FAILED tests/test_magic.py::TestContextManagers::test_hash_class - AttributeError: test_magic FAILED tests/test_magic.py::TestContextManagers::test_hash_obj - AttributeError: test_magic FAILED tests/test_netref_hierachy.py::Test_Netref_Hierarchy::test_instancecheck_across_connections - _get_exception_class..Derived: No module named 'test_magic' FAILED tests/test_service_pickle.py::TestServicePickle::test_dataframe_pickling - NameError: name 'cfg_tests' is not defined =============================================================== 6 failed, 76 passed, 12 skipped, 1 error in 69.26s (0:01:09) ================================================================ ```

Looks like more fixes around imports are needed.

kloczek commented 7 months ago

Here is improved version of above patch.

--- a/tests/test_service_pickle.py
+++ b/tests/test_service_pickle.py
@@ -4,7 +4,7 @@
 import timeit
 import rpyc
 import unittest
-import cfg_tests
+from tests import cfg_tests
 try:
     import pandas as pd
     import numpy as np
--- a/tests/test_affinity.py
+++ b/tests/test_affinity.py
@@ -1,7 +1,7 @@
 import sys
 import time
 import unittest
-import support
+from tests import support
 import rpyc

--- a/tests/test_attributes.py
+++ b/tests/test_attributes.py
@@ -26,7 +26,7 @@
         self.conn.close()

     def test_properties(self):
-        p = self.conn.modules["test_attributes"].Properties()
+        p = self.conn.modules["tests.test_attributes"].Properties()
         print(p.counter)  # 1
         print(p.counter)  # 2
         print(p.counter)  # 3
--- a/tests/test_classic.py
+++ b/tests/test_classic.py
@@ -52,7 +52,7 @@
         self.assertEqual(conn.eval("2+3"), 5)

     def test_modules(self):
-        self.assertIn('test_magic', self.conn.modules)
+        self.assertIn('tests.test_magic', self.conn.modules)
         self.assertNotIn('test_badmagic', self.conn.modules)
         self.assertIsNone(self.conn.builtins.locals()['self']._last_traceback)

--- a/tests/test_magic.py
+++ b/tests/test_magic.py
@@ -37,7 +37,7 @@

     def test_hash_class(self):
         hesh = self.conn.builtins.hash
-        mod = self.conn.modules.test_magic
+        mod = self.conn.modules.tests.test_magic
         self.assertEqual(hash(mod.Base), 4321)
         self.assertEqual(hash(mod.Foo), 4321)
         self.assertEqual(hash(mod.Bar), 4321)
@@ -53,7 +53,7 @@

     def test_hash_obj(self):
         hesh = self.conn.builtins.hash
-        mod = self.conn.modules.test_magic
+        mod = self.conn.modules.tests.test_magic
         obj = mod.Base()

         self.assertNotEqual(hash(obj), 1234)
--- a/tests/test_netref_hierachy.py
+++ b/tests/test_netref_hierachy.py
@@ -108,15 +108,15 @@

     def test_instancecheck_across_connections(self):
         self.conn2 = rpyc.classic.connect('localhost', port=18878)
-        self.conn.execute('import test_magic')
-        self.conn2.execute('import test_magic')
-        foo = self.conn.modules.test_magic.Foo()
-        bar = self.conn.modules.test_magic.Bar()
-        self.assertTrue(isinstance(foo, self.conn.modules.test_magic.Foo))
-        self.assertTrue(isinstance(bar, self.conn2.modules.test_magic.Bar))
-        self.assertFalse(isinstance(bar, self.conn.modules.test_magic.Foo))
+        self.conn.execute('import tests.test_magic')
+        self.conn2.execute('import tests.test_magic')
+        foo = self.conn.modules.tests.test_magic.Foo()
+        bar = self.conn.modules.tests.test_magic.Bar()
+        self.assertTrue(isinstance(foo, self.conn.modules.tests.test_magic.Foo))
+        self.assertTrue(isinstance(bar, self.conn2.modules.tests.test_magic.Bar))
+        self.assertFalse(isinstance(bar, self.conn.modules.tests.test_magic.Foo))
         with self.assertRaises(TypeError):
-            isinstance(self.conn.modules.test_magic.Foo, bar)
+            isinstance(self.conn.modules.tests.test_magic.Foo, bar)

     def test_classic(self):
         x = self.conn.builtin.list((1, 2, 3, 4))

With above pytest still fails with:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0 configfile: pyproject.toml plugins: timeout-2.3.1 collected 95 items tests/test_affinity.py . [ 1%] tests/test_async.py ..... [ 6%] tests/test_attr_access.py ......... [ 15%] tests/test_attributes.py . [ 16%] tests/test_brine.py . [ 17%] tests/test_classic.py ..... [ 23%] tests/test_context_managers.py .. [ 25%] tests/test_custom_service.py ....... [ 32%] tests/test_dataclass.py . [ 33%] tests/test_deploy.py .... [ 37%] tests/test_gdb.py F [ 38%] tests/test_get_id_pack.py .... [ 43%] tests/test_gevent_server.py ... [ 46%] tests/test_ipv6.py s [ 47%] tests/test_magic.py .. [ 49%] tests/test_netref_hierachy.py ........... [ 61%] tests/test_oneshot_server.py . [ 62%] tests/test_race.py . [ 63%] tests/test_refcount.py . [ 64%] tests/test_registry.py ...... [ 70%] tests/test_remote_exception.py .. [ 72%] tests/test_remoting.py s.s.s [ 77%] tests/test_rpyc_over_rpyc.py ... [ 81%] tests/test_service_pickle.py . [ 82%] tests/test_ssh.py EE [ 84%] tests/test_ssl.py . [ 85%] tests/test_teleportation.py ...... [ 91%] tests/test_threaded_server.py ... [ 94%] tests/test_threads.py . [ 95%] tests/test_urllib3.py s [ 96%] tests/test_win32pipes.py sss [100%] ========================================================================================== ERRORS =========================================================================================== __________________________________________________________________________ ERROR at setup of Test_Ssh.test_connect __________________________________________________________________________ cls = @classmethod def setUpClass(cls): if sys.platform == "win32": cls.server = None os.environ["HOME"] = os.path.expanduser("~") else: # assume "ssh localhost" is configured to run without asking for password # `.ssh/config` # Host localhost # HostName 127.0.0.1 # User # IdentityFile cls.server = ThreadedServer(SlaveService, hostname="localhost", ipv6=False, port=18888, auto_register=False) cls.server._start_in_thread() > cls.remote_machine = SshMachine("localhost") tests/test_ssh.py:34: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__ BaseRemoteMachine.__init__( /usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__ self._session = self.session(new_session=new_session) /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:243: in session self.popen( /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:190: in popen return self._ssh_command[tuple(cmdline)].popen(**kwargs) /usr/lib/python3.10/site-packages/plumbum/commands/base.py:332: in popen return self.cmd.popen(self.args + list(args), **kwargs) /usr/lib/python3.10/site-packages/plumbum/machines/local.py:115: in popen return self.machine._popen( /usr/lib/python3.10/site-packages/plumbum/machines/local.py:295: in _popen proc = PlumbumLocalPopen( /usr/lib/python3.10/site-packages/plumbum/machines/local.py:29: in __init__ self._proc = Popen(*args, **kwargs) # pylint: disable=consider-using-with /usr/lib64/python3.10/subprocess.py:971: in __init__ self._execute_child(args, executable, preexec_fn, close_fds, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh'], executable = b'/usr/bin/ssh', preexec_fn = None close_fds = True, pass_fds = (), cwd = '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0' env = {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-...cc1 -flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security', ...} startupinfo = None, creationflags = 0, shell = False, p2cread = 14, p2cwrite = 17, c2pread = 19, c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True, gid = None, gids = None uid = None, umask = -1, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] elif isinstance(args, os.PathLike): if shell: raise TypeError('path-like args is not allowed when ' 'shell is true') args = [args] else: args = list(args) if shell: # On Android the default shell is at '/system/bin/sh'. unix_shell = ('/system/bin/sh' if hasattr(sys, 'getandroidapilevel') else '/bin/sh') args = [unix_shell, "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] sys.audit("subprocess.Popen", executable, args, cwd, env) if (_USE_POSIX_SPAWN and os.path.dirname(executable) and preexec_fn is None and not close_fds and not pass_fds and cwd is None and (p2cread == -1 or p2cread > 2) and (c2pwrite == -1 or c2pwrite > 2) and (errwrite == -1 or errwrite > 2) and not start_new_session and gid is None and gids is None and uid is None and umask < 0): self._posix_spawn(args, executable, env, restore_signals, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) return orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) > self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, gid, gids, uid, umask, preexec_fn) E AttributeError: 'NoneType' object has no attribute 'fork_exec' /usr/lib64/python3.10/subprocess.py:1796: AttributeError __________________________________________________________________________ ERROR at setup of Test_Ssh.test_simple ___________________________________________________________________________ cls = @classmethod def setUpClass(cls): if sys.platform == "win32": cls.server = None os.environ["HOME"] = os.path.expanduser("~") else: # assume "ssh localhost" is configured to run without asking for password # `.ssh/config` # Host localhost # HostName 127.0.0.1 # User # IdentityFile cls.server = ThreadedServer(SlaveService, hostname="localhost", ipv6=False, port=18888, auto_register=False) cls.server._start_in_thread() > cls.remote_machine = SshMachine("localhost") tests/test_ssh.py:34: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__ BaseRemoteMachine.__init__( /usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__ self._session = self.session(new_session=new_session) /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:243: in session self.popen( /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:190: in popen return self._ssh_command[tuple(cmdline)].popen(**kwargs) /usr/lib/python3.10/site-packages/plumbum/commands/base.py:332: in popen return self.cmd.popen(self.args + list(args), **kwargs) /usr/lib/python3.10/site-packages/plumbum/machines/local.py:115: in popen return self.machine._popen( /usr/lib/python3.10/site-packages/plumbum/machines/local.py:295: in _popen proc = PlumbumLocalPopen( /usr/lib/python3.10/site-packages/plumbum/machines/local.py:29: in __init__ self._proc = Popen(*args, **kwargs) # pylint: disable=consider-using-with /usr/lib64/python3.10/subprocess.py:971: in __init__ self._execute_child(args, executable, preexec_fn, close_fds, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh'], executable = b'/usr/bin/ssh', preexec_fn = None close_fds = True, pass_fds = (), cwd = '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0' env = {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-...cc1 -flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security', ...} startupinfo = None, creationflags = 0, shell = False, p2cread = 14, p2cwrite = 17, c2pread = 19, c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True, gid = None, gids = None uid = None, umask = -1, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] elif isinstance(args, os.PathLike): if shell: raise TypeError('path-like args is not allowed when ' 'shell is true') args = [args] else: args = list(args) if shell: # On Android the default shell is at '/system/bin/sh'. unix_shell = ('/system/bin/sh' if hasattr(sys, 'getandroidapilevel') else '/bin/sh') args = [unix_shell, "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] sys.audit("subprocess.Popen", executable, args, cwd, env) if (_USE_POSIX_SPAWN and os.path.dirname(executable) and preexec_fn is None and not close_fds and not pass_fds and cwd is None and (p2cread == -1 or p2cread > 2) and (c2pwrite == -1 or c2pwrite > 2) and (errwrite == -1 or errwrite > 2) and not start_new_session and gid is None and gids is None and uid is None and umask < 0): self._posix_spawn(args, executable, env, restore_signals, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) return orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) > self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, gid, gids, uid, umask, preexec_fn) E AttributeError: 'NoneType' object has no attribute 'fork_exec' /usr/lib64/python3.10/subprocess.py:1796: AttributeError ========================================================================================= FAILURES ========================================================================================== _____________________________________________________________________________________ Test_GDB.test_gdb _____________________________________________________________________________________ self = def test_gdb(self): print(0) parent_gdb_conn = rpyc.connect(host='localhost', port=18878) print(1) gdb = parent_gdb_conn.root.get_gdb() print(2) > gdb.execute('file {}'.format(self.a_out)) tests/test_gdb.py:63: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rpyc/core/netref.py:239: in __call__ return syncreq(_self, consts.HANDLE_CALL, args, kwargs) rpyc/core/netref.py:63: in syncreq return conn.sync_request(handler, proxy, *args) rpyc/core/protocol.py:744: in sync_request return _async_res.value _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = @property def value(self): """Returns the result of the operation. If the result has not yet arrived, accessing this property will wait for it. If the result does not arrive before the expiry time elapses, :class:`AsyncResultTimeout` is raised. If the returned result is an exception, it will be raised here. Otherwise, the result is returned directly. """ self.wait() if self._is_exc: > raise self._obj E _get_exception_class..Derived: connection closed by peer E E ========= Remote Traceback (1) ========= E Traceback (most recent call last): E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request E res = self._HANDLERS[handler](self, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call E return obj(*args, **dict(kwargs)) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/netref.py", line 239, in __call__ E return syncreq(_self, consts.HANDLE_CALL, args, kwargs) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/netref.py", line 63, in syncreq E return conn.sync_request(handler, proxy, *args) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 744, in sync_request E return _async_res.value E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/async_.py", line 109, in value E self.wait() E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/async_.py", line 51, in wait E self._conn.serve(self._ttl, waiting=self._waiting) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 464, in serve E data = self._channel.poll(timeout) and self._channel.recv() E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/channel.py", line 55, in recv E header = self.stream.read(self.FRAME_HEADER.size) E File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/stream.py", line 280, in read E raise EOFError("connection closed by peer") E EOFError: connection closed by peer rpyc/core/async_.py:111: EOFError ----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------ 0 1 36539 2 ===================================================================================== warnings summary ====================================================================================== tests/test_gevent_server.py::Test_GeventServer::test_connection /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_gevent_server.py:18: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util.ssl_ (/usr/lib/python3.10/site-packages/urllib3/util/ssl_.py)', 'urllib3.util (/usr/lib/python3.10/site-packages/urllib3/util/__init__.py)']. monkey.patch_all() tests/test_threaded_server.py::Test_ThreadPoolServer::test_connection /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/utils/server.py:349: DeprecationWarning: setName() is deprecated, set the name attribute instead self.polling_thread.setName('PollingThread') -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================== short test summary info ================================================================================== SKIPPED [1] tests/test_ipv6.py:19: requires IPv6 SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available SKIPPED [1] tests/test_win32pipes.py:11: Requires windows SKIPPED [1] tests/test_win32pipes.py:24: Requires windows SKIPPED [1] tests/test_win32pipes.py:56: Requires windows ERROR tests/test_ssh.py::Test_Ssh::test_connect - AttributeError: 'NoneType' object has no attribute 'fork_exec' ERROR tests/test_ssh.py::Test_Ssh::test_simple - AttributeError: 'NoneType' object has no attribute 'fork_exec' FAILED tests/test_gdb.py::Test_GDB::test_gdb - _get_exception_class..Derived: connection closed by peer ========================================================= 1 failed, 84 passed, 8 skipped, 2 warnings, 2 errors in 99.44s (0:01:39) ========================================================== ```
kloczek commented 6 months ago

I've added https://github.com/tomerfiliba-org/rpyc/pull/557

kloczek commented 6 months ago

After send prod build request to my build infra I found that units which requires running ssh server on local hosts are failing

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' --deselect tests/test_gdb.py::Test_GDB::test_gdb ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0 configfile: pyproject.toml collected 95 items / 1 deselected / 94 selected tests/test_affinity.py . [ 1%] tests/test_async.py ..... [ 6%] tests/test_attr_access.py ......... [ 15%] tests/test_attributes.py . [ 17%] tests/test_brine.py . [ 18%] tests/test_classic.py ..... [ 23%] tests/test_context_managers.py .. [ 25%] tests/test_custom_service.py ....... [ 32%] tests/test_dataclass.py . [ 34%] tests/test_deploy.py FFFs [ 38%] tests/test_get_id_pack.py .... [ 42%] tests/test_gevent_server.py sss [ 45%] tests/test_ipv6.py s [ 46%] tests/test_magic.py .. [ 48%] tests/test_netref_hierachy.py ........... [ 60%] tests/test_oneshot_server.py . [ 61%] tests/test_race.py . [ 62%] tests/test_refcount.py . [ 63%] tests/test_registry.py ...... [ 70%] tests/test_remote_exception.py .. [ 72%] tests/test_remoting.py s.s.s [ 77%] tests/test_rpyc_over_rpyc.py ... [ 80%] tests/test_service_pickle.py . [ 81%] tests/test_ssh.py ss [ 84%] tests/test_ssl.py . [ 85%] tests/test_teleportation.py ...... [ 91%] tests/test_threaded_server.py ... [ 94%] tests/test_threads.py . [ 95%] tests/test_urllib3.py s [ 96%] tests/test_win32pipes.py sss [100%] =================================== FAILURES =================================== ________________________ TestDeploy.test_close_timeout _________________________ self = def test_close_timeout(self): expected_timeout = 4 observed_timeouts = [] original_communicate = subprocess.Popen.communicate def replacement_communicate(self, input=None, timeout=None): observed_timeouts.append(timeout) return original_communicate(self, input, timeout) try: subprocess.Popen.communicate = replacement_communicate > rem = SshMachine("localhost") tests/test_deploy.py:47: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__ BaseRemoteMachine.__init__( /usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__ self._session = self.session(new_session=new_session) /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session return ShellSession( /usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__ self._startup_result = self.run("") /usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run return run_proc(self.popen(cmd), retcode) /usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc stdout, stderr = proc.communicate() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = input = None def communicate(self, input=None): # pylint: disable=redefined-builtin """Consumes the process' stdout and stderr until the it terminates. :param input: An optional bytes/buffer object to send to the process over stdin :returns: A tuple of (stdout, stderr) """ stdout = [] stderr = [] sources = [("1", stdout, self.stdout)] if not self.isatty: # in tty mode, stdout and stderr are unified sources.append(("2", stderr, self.stderr)) i = 0 while sources: if input: chunk = input[:1000] self.stdin.write(chunk) self.stdin.flush() input = input[1000:] i = (i + 1) % len(sources) name, coll, pipe = sources[i] try: line = pipe.readline() shell_logger.debug("%s> %r", name, line) except EOFError as err: shell_logger.debug("%s> Nothing returned.", name) self.proc.poll() returncode = self.proc.returncode stdout = b"".join(stdout).decode(self.custom_encoding, "ignore") stderr = b"".join(stderr).decode(self.custom_encoding, "ignore") argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1] if returncode == 5: raise IncorrectLogin( argv, returncode, stdout, stderr, message="Incorrect username or password provided", host=self.host, ) from None if returncode == 6: raise HostPublicKeyUnknown( argv, returncode, stdout, stderr, message="The authenticity of the host can't be established", host=self.host, ) from None if returncode != 0: > raise SSHCommsError( argv, returncode, stdout, stderr, message="SSH communication failed", host=self.host, ) from None E plumbum.machines.session.SSHCommsError: SSH communication failed E Return code: | 255 E Command line: | 'true ' E Host: | localhost E Stderr: | ssh: connect to host localhost port 22: Connection refused /usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError __________________ TestDeploy.test_close_timeout_default_none __________________ self = def test_close_timeout_default_none(self): observed_timeouts = [] original_communicate = subprocess.Popen.communicate def replacement_communicate(self, input=None, timeout=None): observed_timeouts.append(timeout) return original_communicate(self, input, timeout) try: subprocess.Popen.communicate = replacement_communicate > rem = SshMachine("localhost") tests/test_deploy.py:71: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__ BaseRemoteMachine.__init__( /usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__ self._session = self.session(new_session=new_session) /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session return ShellSession( /usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__ self._startup_result = self.run("") /usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run return run_proc(self.popen(cmd), retcode) /usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc stdout, stderr = proc.communicate() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = input = None def communicate(self, input=None): # pylint: disable=redefined-builtin """Consumes the process' stdout and stderr until the it terminates. :param input: An optional bytes/buffer object to send to the process over stdin :returns: A tuple of (stdout, stderr) """ stdout = [] stderr = [] sources = [("1", stdout, self.stdout)] if not self.isatty: # in tty mode, stdout and stderr are unified sources.append(("2", stderr, self.stderr)) i = 0 while sources: if input: chunk = input[:1000] self.stdin.write(chunk) self.stdin.flush() input = input[1000:] i = (i + 1) % len(sources) name, coll, pipe = sources[i] try: line = pipe.readline() shell_logger.debug("%s> %r", name, line) except EOFError as err: shell_logger.debug("%s> Nothing returned.", name) self.proc.poll() returncode = self.proc.returncode stdout = b"".join(stdout).decode(self.custom_encoding, "ignore") stderr = b"".join(stderr).decode(self.custom_encoding, "ignore") argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1] if returncode == 5: raise IncorrectLogin( argv, returncode, stdout, stderr, message="Incorrect username or password provided", host=self.host, ) from None if returncode == 6: raise HostPublicKeyUnknown( argv, returncode, stdout, stderr, message="The authenticity of the host can't be established", host=self.host, ) from None if returncode != 0: > raise SSHCommsError( argv, returncode, stdout, stderr, message="SSH communication failed", host=self.host, ) from None E plumbum.machines.session.SSHCommsError: SSH communication failed E Return code: | 255 E Command line: | 'true ' E Host: | localhost E Stderr: | ssh: connect to host localhost port 22: Connection refused /usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError ____________________________ TestDeploy.test_deploy ____________________________ self = def test_deploy(self): > rem = SshMachine("localhost") tests/test_deploy.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__ BaseRemoteMachine.__init__( /usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__ self._session = self.session(new_session=new_session) /usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session return ShellSession( /usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__ self._startup_result = self.run("") /usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run return run_proc(self.popen(cmd), retcode) /usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc stdout, stderr = proc.communicate() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = input = None def communicate(self, input=None): # pylint: disable=redefined-builtin """Consumes the process' stdout and stderr until the it terminates. :param input: An optional bytes/buffer object to send to the process over stdin :returns: A tuple of (stdout, stderr) """ stdout = [] stderr = [] sources = [("1", stdout, self.stdout)] if not self.isatty: # in tty mode, stdout and stderr are unified sources.append(("2", stderr, self.stderr)) i = 0 while sources: if input: chunk = input[:1000] self.stdin.write(chunk) self.stdin.flush() input = input[1000:] i = (i + 1) % len(sources) name, coll, pipe = sources[i] try: line = pipe.readline() shell_logger.debug("%s> %r", name, line) except EOFError as err: shell_logger.debug("%s> Nothing returned.", name) self.proc.poll() returncode = self.proc.returncode stdout = b"".join(stdout).decode(self.custom_encoding, "ignore") stderr = b"".join(stderr).decode(self.custom_encoding, "ignore") argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1] if returncode == 5: raise IncorrectLogin( argv, returncode, stdout, stderr, message="Incorrect username or password provided", host=self.host, ) from None if returncode == 6: raise HostPublicKeyUnknown( argv, returncode, stdout, stderr, message="The authenticity of the host can't be established", host=self.host, ) from None if returncode != 0: > raise SSHCommsError( argv, returncode, stdout, stderr, message="SSH communication failed", host=self.host, ) from None E plumbum.machines.session.SSHCommsError: SSH communication failed E Return code: | 255 E Command line: | 'true ' E Host: | localhost E Stderr: | ssh: connect to host localhost port 22: Connection refused /usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError =============================== warnings summary =============================== tests/test_threaded_server.py::Test_ThreadPoolServer::test_connection /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/utils/server.py:349: DeprecationWarning: setName() is deprecated, set the name attribute instead self.polling_thread.setName('PollingThread') -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ SKIPPED [1] tests/test_deploy.py:84: Paramiko is not available SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available SKIPPED [1] tests/test_ipv6.py:19: requires IPv6 SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm SKIPPED [1] tests/test_ssh.py:53: Requires paramiko_machine to localhost SKIPPED [1] tests/test_ssh.py:48: Requires paramiko_machine to localhost SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available SKIPPED [1] tests/test_win32pipes.py:11: Requires windows SKIPPED [1] tests/test_win32pipes.py:24: Requires windows SKIPPED [1] tests/test_win32pipes.py:56: Requires windows FAILED tests/test_deploy.py::TestDeploy::test_close_timeout - plumbum.machine... FAILED tests/test_deploy.py::TestDeploy::test_close_timeout_default_none - pl... FAILED tests/test_deploy.py::TestDeploy::test_deploy - plumbum.machines.sessi... = 3 failed, 77 passed, 14 skipped, 1 deselected, 1 warning in 90.22s (0:01:30) = ```

I think that it would be better if test suite in session will start sshd on some high port and use it to test module remote calls.