search

debasishg / scala-redis

A scala library for connecting to a redis server, or a cluster of redis nodes using consistent hashing on the client side.
1.02k stars 219 forks source link

Parallel calls with a single client in redis 3 #167

Closed giftig closed 5 years ago

giftig commented 8 years ago

I've noticed a significant difference between the behaviour of this library when making parallel redis calls using a single client. Perhaps that's a sign I simply shouldn't be doing that, but I suspect it's more to do with changes to the way such calls are handled in redis 3 and the client falling behind.

I'm doing parallel calls a bit like this:

    ...
    keys flatMap {
      keys: List[String] => Future.traverse(keys) {
        k: String => _switchConfig(k, redisClient) map { conf: Condition => (k, conf) }
      }
    }

Where _switchConfig does a redis GET using the redisClient it's given (paraphrased from my "flippy" project: https://www.github.com/giftig/flippy). I'm using this to grab a load of JSON objects from a batch of redis keys and send the objects back to the client. When I test it against redis 2.8.11, it works fine, every time, but when using 3.2.0 I get a garbled response which seems to be the result of getting two results spliced into each other, which the library then seems to misinterpret as a cursor value and breaks pretty badly. Here's what a good and bad request to my service looks like:

flippy (master) $ docker start redis_2.8
redis_2.8
flippy (master) $ ./src/test/resources/scripts/bootstrap &> /dev/null
flippy (master) $
flippy (master) $
flippy (master) $ curl localhost:9001/switch/; echo ''
[{"name":"is_hodor","condition":{"condition_type":"namespaced","attr":"name","condition":{"condition_type":"equals","value":"Hodor"},"fallback":false}},{"name":"is_staff","condition":{"condition_type":"namespaced","attr":"is_staff","condition":{"condition_type":"equals","value":true},"fallback":false}}]
flippy (master) $ docker stop redis_2.8 && docker start redis
redis_2.8
redis
flippy (master) $ ./src/test/resources/scripts/bootstrap &> /dev/null
flippy (master) $
flippy (master) $
flippy (master) $ curl localhost:9001/switch/; echo ''
{"success":false,"reason":"An internal error occurred: java.lang.NumberFormatException: For input string: \"18\n{\"ondition_ye\"acd,atr:nm\"\"odto\"{cniintp\"\"qas,vle:Hdr}\"alak:as}\n$19\""}

I've used redis fairly extensively in the past, and I'm fairly sure it's meant to be ok to do async calls with a single connection like this, which is reinforced by the fact that it's working fine with this library in 2.8, so I suspect this is a bug in the library; I haven't found any documentation indicating recommendations around this have changed in redis 3, either.

I've worked around this problem by using a new redis connection for each command, to ensure it works with 3.x

giftig commented 8 years ago

For the record, I'm using:

    <dependency>
      <groupId>net.debasishg</groupId>
      <artifactId>redisclient_2.10</artifactId>
      <version>3.0</version>
      <optional>true</optional>
    </dependency>