leporo / tornado-redis

Asynchronous Redis client that works within Tornado IO loop.
666 stars 162 forks source link

Pipeline destroys a connection in the pool #34

Closed thlawiczka closed 11 years ago

thlawiczka commented 11 years ago

Lets start with an example:

from tornado import gen
import tornadoredis

POOL = tornadoredis.ConnectionPool(...)

@gen.coroutine
def func():
    client = tornadoredis.Client(connection_pool=POOL, selected_db=0)
    with client.pipeline() as pipe:
          pipe.get('xxx')
          # ....
          yield gen.Task(pipe.execute)
    yield gen.Task(client.disconnect)

Generally it's working, but the connection is destroyed unnecessarily. The next operation takes a connection from the pool and fails with an exception "Tried to read from non-existent connection".

It happens because the Client.pipeline() shares a connection with a new Pipeline object, which also is a Client. Therefore there are two clients using this same Connection object. At the end of the scope, the original Client releases a connection to the pool (via .del()). The Pipeline object also uses a Client.del() but have on information about the pool, so it calls a disconnect().

To solve this, you can simply add an empty impl of .del() to the Pipeline class and expect that creator of the Pipeline will take care about the connection.

Regards, Tom

leporo commented 11 years ago

Thank you. I added an empty implementation of del method as you suggested.

Please check does this change work for you?