basho / riak-python-client

The Riak client for Python.
Apache License 2.0
321 stars 183 forks source link

Use of RiakClient after close() leaks connections [JIRA: CLIENTS-1024] #500

Closed kesslerm closed 7 years ago

kesslerm commented 7 years ago

If after RiakClient.close() has been called any other operation that needs a http/tcp connection is invoked (such as RiakClient.get() or RiakClient.put()) the http/tcp pools re-acquire a new connection resource automatically, but subsequent calls to RiakClient.close() do not release that resource. This is due to the guard flag self._closed, which is set on the first call to RiakClient.close(). The connections acquired this way are not closed until they either timeout when unused (if keep_alive is not set), or the python process terminates.

Can the guard flag simply be removed, or would that cause issues with the multiget and multiput pools, which are stopped as part of the close operation?

The behaviour was introduced in commit 32d19da6. Prior to that one could simply call RiakClient.close() to force the client to re-establish a new connection. This is useful with PBC connections to load balancers such as HAProxy, which otherwise will not be able to rebalance the long living PBC connections.

lukebakken commented 7 years ago

@kesslerm - after close() is called that client instance should not be able to service further requests and should throw an exception if one is tried. Does that sound reasonable?

kesslerm commented 7 years ago

@lukebakken That's one reasonable semantic, no use after close(). However, I had originally employed the use after close() pattern to overcome the issue that there was no other way to force the PBC / tcp connections to reconnect. I will follow up with another issue for that.

lukebakken commented 7 years ago

@kesslerm - to force reconnection, you should close() the current client instance, then instantiate a new client instance. Re-use after close() is not supported in any client (I think...)