Multiple thread problem?

[muyinliu]

Use CL-Redis in Hunchentoot, while multiple request happened, an error occur:

Redis error: NIL
ERR unknown command '$12'
    [Condition of type REDIS:REDIS-ERROR-REPLY]

enter the break loop while `(red:get "@@P_name_109")` and

Redis error: NIL
ERR unknown command '@@P_name_109'
    [Condition of type REDIS:REDIS-ERROR-REPLY]

enter the break loop while `(red:dismember "@@P_id" 784)` So I think it is a multiple thread problem. Should I use more than 1 connection for Redis? How to solve this problem? Thanks.

[vseloved]

Hi, such things usually happen in 2 situations:

• when you try to use the same connection from multiple threads
• or when you abort a request improperly (for instance, on timeout or some other external condition)

For the first situation the solution is simple: wrap your calls in with-connection; or if you don't want to create a fresh connection for each request, use some pool of connections.

For the second situation the solution will be either to close the connection in the error handler, or to defer a condition and handle it after the redis request is complete.

Without seeing the actual code that interacts with redis I can't add much to this.

[muyinliu]

Thanks for your advise!
I decide to try with-connection with pool of connections. Conditions are not easy to handle. What you did is great, thanks again! (I'll reply again after I try)

[vseloved]

Sorry, maybe I was not clear enough, but using with-connection and using a pool of connections is exclusive.

If you use with-connection all the code inside this form will access the connection established by it, so if you wrap each of your HTTP handlers that access redis in with-connection the interaction with redis will be thread-safe

A connection pool, is a more complicated approach, and you have to either use a library (although, I'm not aware of the general connection pool library for CL at the moment) or roll your own solution. Schematically, it will work this way:

1. You intialize a number of connection
2. When you want to access redis, you bind redis:*connection* to one of the idle connections from the pool, and take it from the pool
3. When you end your redis session you have to return the connection to the pool (ideally, in something like unwind-protect to ensure, that you don't loose connection if there's some error etc.)

See how postomodern library, which handles connections very similar to cl-redis, does it: https://github.com/marijnh/Postmodern/blob/master/postmodern/connect.lisp

[muyinliu]

Good news! After using the solution, the multiple thread problem solved! I will pull a request about connection.lisp with connection pool.

Thanks a lot!

[muyinliu]

I pull a request just now: https://github.com/vseloved/cl-redis/pull/20 Please check.