[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

[shiladitya146]

I get the given error message when running the code. Not sure how to address this

conn = pyam.iiasa.Connection() conn.valid_connections

[danielhuppmann]

@meksor @peterkolp, do you have any idea whether this is an issue with the IIASA certificates? Or can you chime in which type of information from the user would be helpful to identify the solution?

[peterkolp]

I've no clue ..,

[shiladitya146]

I tried a few things but still the same error. Here is the complete error message: SSLError: HTTPSConnectionPool(host='db1.ene.iiasa.ac.at', port=443): Max retries exceeded with url: /EneAuth/config/v1/anonym (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

[danielhuppmann]

@meksor identified an issue with the token-timeout in the IIASA authentication system yesterday and implemented a fix. Please try again.

[danielhuppmann]

Can you please try again and let us know if the problem still exists?

[stella-ds]

@danielhuppmann I'm getting a similar issue with MaxRetryError: HTTPSConnectionPool(host='db1.ene.iiasa.ac.at', port=443): Max retries exceeded with url: /EneAuth/config/v1/anonym (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)')))

environment

• pyopenssl==22.0.0
• pyam ==1.5.0

[danielhuppmann]

Thank you for reporting this error, I can reproduce the issue and we'll investigate.

In the meantime, can you try to install the latest from GitHub using

pip install -e git+https://github.com/IAMconsortium/pyam.git#egg=pyam

and let us know if this fixes your problem?

[stella-ds]

thanks @danielhuppmann! this works pip install git+https://github.com/IAMconsortium/pyam.git

[stella-ds]

@danielhuppmann now encountered the SSL issue again when calling

conn = pyam.iiasa.Connection('engage')

tried a few other connection name same issue

---------------------------------------------------------------------------
SSLCertVerificationError                  Traceback (most recent call last)
/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    702             # Make the request on the httplib connection object.
--> 703             httplib_response = self._make_request(
    704                 conn,

/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    385         try:
--> 386             self._validate_conn(conn)
    387         except (SocketTimeout, BaseSSLError) as e:

/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
   1041         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
-> 1042             conn.connect()
   1043 

/opt/conda/lib/python3.8/site-packages/urllib3/connection.py in connect(self)
    413 
--> 414         self.sock = ssl_wrap_socket(
    415             sock=conn,

/opt/conda/lib/python3.8/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data, tls_in_tls)
    448     if send_sni:
--> 449         ssl_sock = _ssl_wrap_socket_impl(
    450             sock, context, tls_in_tls, server_hostname=server_hostname

/opt/conda/lib/python3.8/site-packages/urllib3/util/ssl_.py in _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname)
    492     if server_hostname:
--> 493         return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
    494     else:

/opt/conda/lib/python3.8/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    499         # ctx._wrap_socket()
--> 500         return self.sslsocket_class._create(
    501             sock=sock,

/opt/conda/lib/python3.8/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040                     self.do_handshake()
   1041             except (OSError, ValueError):

/opt/conda/lib/python3.8/ssl.py in do_handshake(self, block)
   1308                 self.settimeout(None)
-> 1309             self._sslobj.do_handshake()
   1310         finally:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
/opt/conda/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    488             if not chunked:
--> 489                 resp = conn.urlopen(
    490                     method=request.method,

/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    786 
--> 787             retries = retries.increment(
    788                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

/opt/conda/lib/python3.8/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    591         if new_retry.is_exhausted():
--> 592             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    593 

MaxRetryError: HTTPSConnectionPool(host='db1.ene.iiasa.ac.at', port=443): Max retries exceeded with url: /engage-api/rest/v2.1/runs?getOnlyDefaultRuns=true&includeMetadata=false (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)')))

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
/tmp/ipykernel_1734/3917936776.py in <module>
----> 1 models = conn.models()

/opt/conda/lib/python3.8/site-packages/pyam/iiasa.py in models(self)
    333     def models(self):
    334         """List all models in the connected resource"""
--> 335         return pd.Series(self._query_index()["model"].unique(), name="model")
    336 
    337     def scenarios(self):

/opt/conda/lib/python3.8/site-packages/pyam/iiasa.py in _query_index(self, default, meta)
    273         add_url = f"runs?getOnlyDefaultRuns={_default}&includeMetadata={_meta}"
    274         url = "/".join([self._base_url, add_url])
--> 275         r = requests.get(url, headers=self.auth())
    276         _check_response(r)
    277 

/opt/conda/lib/python3.8/site-packages/requests/api.py in get(url, params, **kwargs)
     71     """
     72 
---> 73     return request("get", url, params=params, **kwargs)
     74 
     75 

/opt/conda/lib/python3.8/site-packages/requests/api.py in request(method, url, **kwargs)
     57     # cases, and look like a memory leak in others.
     58     with sessions.Session() as session:
---> 59         return session.request(method=method, url=url, **kwargs)
     60 
     61 

/opt/conda/lib/python3.8/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    585         }
    586         send_kwargs.update(settings)
--> 587         resp = self.send(prep, **send_kwargs)
    588 
    589         return resp

/opt/conda/lib/python3.8/site-packages/requests/sessions.py in send(self, request, **kwargs)
    699 
    700         # Send the request
--> 701         r = adapter.send(request, **kwargs)
    702 
    703         # Total elapsed time of the request (approximately)

/opt/conda/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    561             if isinstance(e.reason, _SSLError):
    562                 # This branch is for urllib3 v1.22 and later.
--> 563                 raise SSLError(e, request=request)
    564 
    565             raise ConnectionError(e, request=request)

SSLError: HTTPSConnectionPool(host='db1.ene.iiasa.ac.at', port=443): Max retries exceeded with url: /engage-api/rest/v2.1/runs?getOnlyDefaultRuns=true&includeMetadata=false (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)')))

[danielhuppmann]

The IIASA ICT department issued a new SSL certificate for the offending server, all functions work again as expected (confirmed by the nightly tests, see https://github.com/IAMconsortium/pyam/actions/runs/2865409684).