aws / amazon-redshift-python-driver

Redshift Python Connector. It supports Python Database API Specification v2.0.
Apache License 2.0
202 stars 72 forks source link

fix(connection): unpack_from error caused by network issues #185

Closed soksamnanglim closed 10 months ago

soksamnanglim commented 10 months ago

Description

Motivation and Context

Problem: When the server side socket closes due to the network issues, redshift-connector raises an unpack_from error that is confusing and misleading to users. Now, when the server side socket closes due to network issues, redshift-connector will raise the error suggesting users to double check their client-side networking configurations, and if they set a timeout for their connection, will ask them to raise it or use None as default.

The fix address this issue. (due to network issues)

Testing

Wrote unit and integration tests. Ran manual tests using default timeout and manually setting timeout:

Deterministic example:

conn = redshift_connector.connect(dbkwargs)
conn2 = redshift_connector.connect(dbkwargs)
res = conn2.run("select pg_backend_pid()")
pid1 = res[0][0]

conn.run("select pg_terminate_backend(:v)", v=pid1)
conn2.run("select 1") 

Nondeterministic example:

conn = redshift_connector.connect(dbkwargs)
cursor = conn.cursor()
try:
   for i in range(100):
      print(f"Connection is open. Waiting... {i} round")
      time.sleep(33)
      cursor.execute("select * from svv_all_columns")
     time.sleep(17)

except Exception as e:
   raise e
finally:
   cursor.close()

Screenshots (if appropriate)

Types of changes

Checklist

soksamnanglim commented 10 months ago

fixed the unit tests, added error handling logic to handle_messages_merge_socket_read and unit tests for that as well.