scylladb / python-driver

ScyllaDB Python Driver, originally DataStax Python Driver for Apache Cassandra
https://python-driver.docs.scylladb.com
Apache License 2.0
70 stars 42 forks source link

Memory leak when `connect` fails with exception AuthenticationFailed #318

Open robertoz-01 opened 3 months ago

robertoz-01 commented 3 months ago

Every time a connection is attempted and fails with AuthenticationFailed, the memory used by the process increases.

For example, running this code:

from cassandra.cluster import Cluster

keyspace = "kkkk"
node = '10.10.10.10'

session = None
while session is None:
    try:
        scylla_cluster = Cluster([node])

        session = scylla_cluster.connect(keyspace)
    except Exception as e:
        print(e)

print("Session opened")

With a scylla node that requires authentication, this will repeatedly fail with the error message ('Unable to connect to any servers', {'10.11.12.3:9042': AuthenticationFailed('Remote end requires authentication')}) and the process will continue to increase its memory usage (on my machine it gets to around 1GB of memory usage in 2 minutes).

This is not happening in case the connection fails for another reason like, for example, ConnectionRefusedError.

Trying to debug the issue with Pympler, I can see all the objects of basic types increasing at each reconnection:

                                                types |   # objects |   total size
===================================================== | =========== | ============
                                                 list |      179515 |    167.57 MB
                                                  str |      197414 |     14.08 MB
                                                 dict |       32454 |     10.31 MB
                                    collections.deque |        8813 |      9.55 MB
                                                  int |      140409 |      3.77 MB