Python 3.10 in Spack causes pymongo to fail

[mtitov]

Spack started to use python 3.10 as a default (since 0.19 release), and that causes issues with pymongo

Example of running CI at LLNL (running on Lassen)

  File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-
sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/topology.py", line 218, 
in _select_servers_loop

    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed: any.apps.czapps.llnl.gov:30890: 
EOF occurred in violation of protocol (_ssl.c:997), Timeout: 30s, Topology Description: <TopologyDescription id: 
636bb755cf8c84f11393f1e0, topology_type: Single, servers: [<ServerDescription ('any.apps.czapps.llnl.gov', 30890) 
server_type: Unknown, rtt: None, error=AutoReconnect('SSL handshake failed: any.apps.czapps.llnl.gov:30890: 
EOF occurred in violation of protocol (_ssl.c:997)')>]>

``` [39m================================================================================  Getting Started (RP version 1.16.0) ================================================================================  /usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-paramiko-2.7.1-gzlzhr5zhyshnj654e7iasptpkclkvn7/lib/python3.10/site-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, new session: [rp.session.lassen571.titov1.019305.0004] \ database : [mongodb+ssl://exasdkci:****@any.apps.czapps.llnl.gov:30890/exasdkci?tlsAllowInvalidCertificates=true] err Traceback (most recent call last): File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-radical-pilot-1.16.0-h6hcpc3qiqqbm4y3zy3dhzig6pjntyde/lib/python3.10/site-packages/radical/pilot/session.py", line 181, in _initialize_primary self._dbs = DBSession(sid=self.uid, dburl=dburl, File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-radical-pilot-1.16.0-h6hcpc3qiqqbm4y3zy3dhzig6pjntyde/lib/python3.10/site-packages/radical/pilot/db/database.py", line 49, in __init__ self._mongo, self._db, _, _, _ = ru.mongodb_connect(str(dburl)) File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-radical-utils-1.16.0-2yqu477ih3tyhaxhvw3cve2no6eetfoi/lib/python3.10/site-packages/radical/utils/misc.py", line 137, in mongodb_connect db.authenticate(user, pwd) File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/database.py", line 1578, in authenticate self.client._cache_credentials( File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/mongo_client.py", line 809, in _cache_credentials server = self._get_topology().select_server( File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/topology.py", line 244, in select_server return random.choice(self.select_servers(selector, File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/topology.py", line 202, in select_servers server_descriptions = self._select_servers_loop( File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-pymongo-3.12.1-sxtqohabmc5krn4dngjzwoyzneuviaab/lib/python3.10/site-packages/pymongo/topology.py", line 218, in _select_servers_loop raise ServerSelectionTimeoutError( pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed: any.apps.czapps.llnl.gov:30890: EOF occurred in violation of protocol (_ssl.c:997), Timeout: 30s, Topology Description: ]> The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/WS1/titov1/gitlab-runner/builds/mZQGxXhb/000/gitlab/Exaworks/SDK/radical.pilot/./examples/00_getting_started.py", line 35, in session = rp.Session() File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-radical-pilot-1.16.0-h6hcpc3qiqqbm4y3zy3dhzig6pjntyde/lib/python3.10/site-packages/radical/pilot/session.py", line 153, in __init__ self._initialize_primary(dburl) File "/usr/WS1/exaworks/sdk/spack/opt/spack/linux-rhel7-ppc64le/gcc-9.4.0/py-radical-pilot-1.16.0-h6hcpc3qiqqbm4y3zy3dhzig6pjntyde/lib/python3.10/site-packages/radical/pilot/session.py", line 196, in _initialize_primary raise RuntimeError ('session create failed [%s]' % RuntimeError: session create failed [mongodb+ssl://exasdkci:****@any.apps.czapps.llnl.gov:30890/exasdkci?tlsAllowInvalidCertificates=true] ```

RP v.1.16

[mturilli]

@mtitov Is this still the case with 1.22?

[mtitov]

that wasn't re-tested, would propose to have it for the next sprint

[mtitov]

RP 1.40 release (no MongoDB requirement)