ronf / asyncssh

AsyncSSH is a Python package which provides an asynchronous client and server implementation of the SSHv2 protocol on top of the Python asyncio framework.
Eclipse Public License 2.0
1.54k stars 151 forks source link

2.17.0: pytest fails in `tests/test_pkcs11.py::_TestPKCS11Auth::test_pkcs11_load_keys` unit #693

Open kloczek opened 1 day ago

kloczek commented 1 day ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-asyncssh-2.17.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-asyncssh-2.17.0-2.fc37.x86_64/usr/lib/python3.10/site-packages + /usr/bin/pytest -ra -m 'not network' ============================= test session starts ============================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 rootdir: /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0 configfile: tox.ini testpaths: tests collected 1608 items tests/test_agent.py sssssssssssssssssss [ 1%] tests/test_asn1.py . [ 1%] tests/test_auth.py ........ [ 1%] tests/test_auth_keys.py ......... [ 2%] tests/test_channel.py s...s............................................. [ 5%] ........................................................ [ 8%] tests/test_compression.py . [ 8%] tests/test_config.py ................................................... [ 12%] ......... [ 12%] tests/test_connection.py ...................................ss.......... [ 15%] ........................................................................ [ 20%] .............................s.......................................... [ 24%] .......... [ 25%] tests/test_connection_auth.py ..............................s........... [ 27%] ....................s..........ss.sssss..s..s........................... [ 32%] ss.sssss..s..s................................s......................... [ 36%] .............................................. [ 39%] tests/test_editor.py .................... [ 40%] tests/test_encryption.py .. [ 40%] tests/test_forward.py .................................................. [ 44%] ........................................................................ [ 48%] .................. [ 49%] tests/test_kex.py ..........s [ 50%] tests/test_known_hosts.py ................ [ 51%] tests/test_logging.py ......... [ 51%] tests/test_mac.py .. [ 52%] tests/test_packet.py .. [ 52%] tests/test_pkcs11.py ..........F. [ 52%] tests/test_process.py .................................................. [ 56%] .................................... [ 58%] tests/test_public_key.py ........s....... [ 59%] tests/test_saslprep.py ....... [ 59%] tests/test_sftp.py ..................................................... [ 62%] ........................................................................ [ 67%] ........................................................................ [ 71%] ........................................................................ [ 76%] ........................................................................ [ 80%] ........................................................................ [ 85%] ........................................................................ [ 89%] ......... [ 90%] tests/test_sk.py ............................. [ 92%] tests/test_stream.py ....................... [ 93%] tests/test_subprocess.py ............ [ 94%] tests/test_tuntap.py sssssssssssss.............. [ 96%] tests/test_x11.py ...................................... [ 98%] tests/test_x509.py ......................... [100%] =================================== FAILURES =================================== ____________________ _TestPKCS11Auth.test_pkcs11_load_keys _____________________ self = @asynctest async def test_pkcs11_load_keys(self): """Test authenticating with explicitly loaded PKCS#11 keys""" for key in asyncssh.load_pkcs11_keys('xxx'): for sig_alg in key.sig_algorithms: sig_alg = sig_alg.decode('ascii') with self.subTest(key=key.get_comment(), sig_alg=sig_alg): > async with self.connect( username='ckey', pkcs11_provider='xxx', client_keys=[key], signature_algs=[sig_alg]): /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/tests/test_pkcs11.py:169: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/misc.py:299: in __aenter__ self._coro_result = await self._coro /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/tests/server.py:314: in connect return await asyncssh.connect(host or self._server_addr, /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/connection.py:8834: in connect return await asyncio.wait_for( /usr/lib64/python3.10/asyncio/tasks.py:408: in wait_for return await fut /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/connection.py:453: in _connect await options.waiter /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/connection.py:1092: in _reap_task task.result() /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/auth.py:343: in _send_signed_request await self.send_request(Boolean(True), /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/auth.py:136: in send_request await self._conn.send_userauth_request(self._method, *args, key=key, /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/connection.py:1948: in send_userauth_request sig = await self._loop.run_in_executor(None, key.sign, data) /usr/lib64/python3.10/concurrent/futures/thread.py:58: in run result = self.fn(*self.args, **self.kwargs) /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/pkcs11.py:91: in sign sig = self._privkey.sign(data, mechanism=mechanisms[sig_algorithm]) /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/tests/pkcs11_stub.py:83: in sign sig = self._priv.sign_raw(data, _hash_algs[mechanism]) /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/public_key.py:546: in sign_raw return self._key.sign(data, hash_name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = data = b'\x00\x00\x00 +\x13\x01B\xb9@\xff1\x0c\xb6\xed\x08\x17\xb0_:\xb1#_\xc6\xb7\x06\xea\xab?H-BhD\x97\xef2\x00\x00\x00\x04...\xf1[:\xd0\x82P\xbb\xf3\x83\xc4\xa0a\xb0\xf47\x16\xe5FHk%\xcbo\':k\xf6\x87\xe2\xf4\x02\x88\xa2\xf2\xac\x00\x00\x00\x00' hash_name = 'sha1' def sign(self, data: bytes, hash_name: str = '') -> bytes: """Sign a block of data""" priv_key = cast('rsa.RSAPrivateKey', self.pyca_key) > return priv_key.sign(data, PKCS1v15(), hashes[hash_name]()) E cryptography.exceptions.UnsupportedAlgorithm: sha1 is not supported by this backend for RSA signing. /home/tkloczko/rpmbuild/BUILD/asyncssh-2.17.0/asyncssh/crypto/rsa.py:136: UnsupportedAlgorithm ------------------------------ Captured log call ------------------------------- INFO asyncssh:logging.py:102 Opening SSH connection to 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1566] Accepted SSH client connection INFO asyncssh:logging.py:102 [conn=1566] Local address: 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1566] Peer address: 127.0.0.1, port 42810 INFO asyncssh:logging.py:102 [conn=1567] Connected to SSH server at 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1567] Local address: 127.0.0.1, port 42810 INFO asyncssh:logging.py:102 [conn=1567] Peer address: 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1567] Beginning auth for user ckey INFO asyncssh:logging.py:102 [conn=1566] Beginning auth for user ckey INFO asyncssh:logging.py:102 [conn=1566] Auth for user ckey succeeded INFO asyncssh:logging.py:102 [conn=1566] Sending server host keys disabled INFO asyncssh:logging.py:102 [conn=1567] Auth for user ckey succeeded INFO asyncssh:logging.py:102 [conn=1567] Closing connection INFO asyncssh:logging.py:102 [conn=1567] Sending disconnect: Disconnected by application (11) INFO asyncssh:logging.py:102 [conn=1567] Connection closed INFO asyncssh:logging.py:102 Opening SSH connection to 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1568] Accepted SSH client connection INFO asyncssh:logging.py:102 [conn=1568] Local address: 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1568] Peer address: 127.0.0.1, port 42812 INFO asyncssh:logging.py:102 [conn=1569] Connected to SSH server at 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1569] Local address: 127.0.0.1, port 42812 INFO asyncssh:logging.py:102 [conn=1569] Peer address: 127.0.0.1, port 38353 INFO asyncssh:logging.py:102 [conn=1569] Beginning auth for user ckey INFO asyncssh:logging.py:102 [conn=1568] Beginning auth for user ckey INFO asyncssh:logging.py:102 [conn=1569] Connection failure: sha1 is not supported by this backend for RSA signing. INFO asyncssh:logging.py:102 [conn=1569] Aborting connection =============================== warnings summary =============================== tests/test_process.py::_TestProcessRedirection::test_stdout_stream_keep_open /usr/lib64/python3.10/pathlib.py:574: RuntimeWarning: coroutine 'Queue.join' was never awaited for a in args: Enable tracemalloc to get traceback where the object was allocated. See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info. -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ SKIPPED [1] tests/test_agent.py:283: ssh-agent not available SKIPPED [1] tests/test_agent.py:270: ssh-agent not available SKIPPED [1] tests/test_agent.py:237: ssh-agent not available SKIPPED [1] tests/test_agent.py:350: ssh-agent not available SKIPPED [1] tests/test_agent.py:307: ssh-agent not available SKIPPED [1] tests/test_agent.py:372: ssh-agent not available SKIPPED [1] tests/test_agent.py:141: ssh-agent not available SKIPPED [1] tests/test_agent.py:147: ssh-agent not available SKIPPED [1] tests/test_agent.py:465: ssh-agent not available SKIPPED [1] tests/test_agent.py:472: ssh-agent not available SKIPPED [1] tests/test_agent.py:165: ssh-agent not available SKIPPED [1] tests/test_agent.py:330: ssh-agent not available SKIPPED [1] tests/test_agent.py:394: ssh-agent not available SKIPPED [1] tests/test_agent.py:154: ssh-agent not available SKIPPED [1] tests/test_agent.py:416: ssh-agent not available SKIPPED [1] tests/test_agent.py:222: ssh-agent not available SKIPPED [1] tests/test_agent.py:196: ssh-agent not available SKIPPED [1] tests/test_agent.py:172: ssh-agent not available SKIPPED [1] tests/test_agent.py:456: ssh-agent not available SKIPPED [1] tests/test_channel.py:852: ssh-agent not available SKIPPED [1] tests/test_channel.py:875: ssh-agent not available SKIPPED [1] tests/test_connection.py:630: Netcat not available SKIPPED [1] tests/test_connection.py:642: Netcat not available SKIPPED [1] tests/test_connection.py:1963: Netcat not available SKIPPED [2] tests/test_connection_auth.py:675: Netcat not available SKIPPED [2] tests/test_connection_auth.py:1178: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1246: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1188: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1206: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1223: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1389: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1235: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1496: ssh-agent not available SKIPPED [2] tests/test_connection_auth.py:1367: ssh-agent not available SKIPPED [1] tests/test_connection_auth.py:1688: ssh-agent not available SKIPPED [1] tests/test_kex.py:540: SNTRUP761 not available SKIPPED [1] tests/test_public_key.py:2285: openssl isn't available SKIPPED [1] tests/test_tuntap.py:477: only run utun tests on macOS SKIPPED [1] tests/test_tuntap.py:572: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:556: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:564: only run utun tests on macOS SKIPPED [1] tests/test_tuntap.py:525: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:547: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:533: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:444: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:460: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:452: only run TapTunOSX tests on macOS SKIPPED [1] tests/test_tuntap.py:469: only run utun tests on macOS SKIPPED [1] tests/test_tuntap.py:505: only run utun tests on macOS SKIPPED [1] tests/test_tuntap.py:490: only run utun tests on macOS FAILED tests/test_pkcs11.py::_TestPKCS11Auth::test_pkcs11_load_keys - cryptog... ====== 1 failed, 1547 passed, 60 skipped, 1 warning in 744.93s (0:12:24) ======= ```
List of installed modules in build env: ```console Package Version ------------------ ----------- aiofiles 24.1.0 asn1crypto 1.5.1 bcrypt 4.2.0 build 1.2.2 cached-property 1.5.2 cffi 1.16.0 cryptography 42.0.8 decorator 5.1.1 distro 1.9.0 exceptiongroup 1.1.3 fido2 1.1.3 gssapi 1.8.3 importlib_metadata 8.0.0 iniconfig 2.0.0 installer 0.7.0 packaging 24.0 pluggy 1.5.0 ply 3.11 pycparser 2.22 pyOpenSSL 24.1.0 pyproject_hooks 1.0.0 pytest 8.2.2 python-dateutil 2.9.0.post0 python-pkcs11 0.7.0 setuptools 75.1.0 tokenize_rt 6.0.0 tomli 2.0.1 typing_extensions 4.12.2 wheel 0.44.0 zipp 3.19.2 ```

Please let me know if you need more details or want me to perform some diagnostics.

ronf commented 1 day ago

Regarding the error related to SHA-1 not being supported, that's caused by the security policies on some systems. I've tried to make changes in other cases to skip SHA-1 in my unit tests to avoid this issue, but I don't know of a way on macOS (my primary testing platform) to trigger this issue. So, there may be places I missed.

If you try the following patch, does it resolve this issue?

diff --git a/tests/test_pkcs11.py b/tests/test_pkcs11.py
index 4044506..45f969d 100644
--- a/tests/test_pkcs11.py
+++ b/tests/test_pkcs11.py
@@ -165,6 +165,9 @@ class _TestPKCS11Auth(_CheckPKCS11Auth):
             for sig_alg in key.sig_algorithms:
                 sig_alg = sig_alg.decode('ascii')

+                if sig_alg == 'ssh-rsa':
+                    continue
+
                 with self.subTest(key=key.get_comment(), sig_alg=sig_alg):
                     async with self.connect(
                             username='ckey', pkcs11_provider='xxx',

Regarding the warning about Queue.join, that seems to be a race condition in the cleanup of one of the unit tests which I see on occasion but haven't tracked down yet. Since it is only a warning, this may be something you can ignore for now, though.

Regarding dropping Python 3.7 support, I've generally not been in a hurry to intentionally force the use of newer Python versions, at least not until "cryptography" drops support. So, I'm not sure I want to make the other changes here. That said, some of the modifications you proposed look like they could be beneficial, and not break Python 3.6 or 3.7 support, and I'd consider those. Were you using some kind of automated tool to generate these?

One change I'd prefer not to include is splitting the "import" lines in the "examples" directory. I do split imports onto their own lines elsewhere in the code, but I intentionally use the comma-separated version of the imports in examples to reduce the amount of space they take up when viewing examples in the top-level AsyncSSH doc page. So, I'd want to remove those changes.