IBM / python-sdk-core

The python-sdk-core repository contains core functionality required by Python code generated by the IBM OpenAPI SDK Generator.
Apache License 2.0
20 stars 27 forks source link

We are are updating our database periodically with python sdk core but after every one or few hours we get this error. #201

Closed ansultan1 closed 2 months ago

ansultan1 commented 3 months ago

The connection failed because the SSL certificate is not valid. To use a self-signed certificate, disable verification of the server's SSL certificate by invoking the set_disable_ssl_verification(True) on your service instance and/ or use the disable_ssl_verification option of the authenticator.

Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1060, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator alert ssl.SSLError: [SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error (_ssl.c:1007)
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator During handling of the above exception, another exception occurred:
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 801, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 594, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='jp-osa.iaas.cloud.ibm.com', port=443): Max retries exceeded with url: /v1/load_balancer/profiles?version=2023-05-02&generation=2&limit=100 (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error (_ssl.c:1007)')))
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator During handling of the above exception, another exception occurred:
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/ibm_cloud_sdk_core/base_service.py", line 315, in send
    response = self.http_client.request(**request, cookies=self.jar, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 620, in send
    raise SSLError(e, request=request)
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator requests.exceptions.SSLError: HTTPSConnectionPool(host='jp-osa.iaas.cloud.ibm.com', port=443): Max retries exceeded with url: /v1/load_balancer/profiles?version=2023-05-02&generation=2&limit=100 (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error (_ssl.c:1007)')))
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator WARNING listing IBM_SNAPSHOTS for clouds/5e66537085e74bc39ea7368790bf7c6b/regions/eu-de
Jul 21 23:58:25 ibmmangossyncinitiator-ddff78bb-6r9w7 ibmmangossyncinitiator WARNING Task failed for IBM_LOAD_BALANCER_PROFILES in clouds/5e66537085e74bc39ea7368790bf7c6b/regions/jp-osa. Reason: HTTPSConnectionPool(host='jp-osa.iaas.cloud.ibm.com', port=443): Max retries exceeded with url: /v1/load_balancer/profiles?version=2023-05-02&generation=2&limit=100 (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error (_ssl.c:1007)'))), Error type: <class 'requests.exceptions.SSLError'> 
pyrooka commented 3 months ago

Hi! This looks like an issue on the server side. I can't see anything in that stacktrace that would imply a problem in the core. However there is one suspicious thing: tlsv1 alert decode error. Although I've never seen this exact issue before, TLS v1.2 is the minimum required version in the core and TLS v1 is not supported, so this might be something you want to check. Let me know if you have more information or further questions!

ansultan1 commented 3 months ago

@pyrooka thanks for response. it works well for all calls .i mean for each resource the call is successful but after few hours it throws this error. so if the issue is with tls it should fail on each call. also the TLS version above 1.2 is used

pyrooka commented 3 months ago

Sorry for the delay! To be honest I have no idea about what could be the cause of the issue, since it happens randomly and only after a few hours. I thought maybe something happens with the load balancer which triggers this exception, but that was just a guess. You can enable debug logging to have more details, perhaps you'll find something useful. Sorry I can't really add anything more, since this is really hard - if not impossible - to reproduce on our side. Since the SSL/TLS related code in the core is really simple, this could come from requests or urllib3. Let me know if you have more details or any questions!

Edit: I found 2 kind of issues that can cause this exception:

  1. The server tries to use a higher version of TLS that the client supports. I'm pretty sure that our clients support TLS v1.3 which is the highest at the time of the writing. Maybe an older version of Python doesn't have this, or the Python interpreter was compiled with a different version of OpenSSL and the support of 1.3 is missing...
  2. The hostname is not correct/changes. This can be an issue only if the load balancer if configured with TLS Passthrough. I'm not sure if that could be an issue...
pyrooka commented 2 months ago

I'm closing this issue for now but feel free to re-open if needed.