Open Jackvie opened 4 years ago
@Jackvie You need to debug this deeper on your side and provide some kind of profiling run of the code to see where you are having this major slowdown. I doubt there is some kind of lock issue per say but what i do expect is that you might get into some timeout issue or similar. The one thing i do not get tho is why you create one pipeline and share it among the different threads. Try to make a seperate pipeline for each thread you create and it might increase your performance a ton and then you keep your client a unique instance across the threads maybe?
@Jackvie I attempted to use your script locally on my laptop, i don't have any performance problems or degredations is any way. The script finished in about 0.17 seconds every run w/o any issues so you need to track down the problem becuase it is probably something other then this client library code at this point.
Pipeline object is not thread-safe as claimed for redis-py: https://github.com/andymccurdy/redis-py#thread-safety
Probably we're having similar limitation here.
@rudyryk What i see there is that it is not smart to create a shared Pipeline object as a singleton and share it across the code and use it at multiple places. I would never do this with the plain redis-py object either so doing this for the cluster version i would not do that either then.
redis-cluster-py version 2.1.0 redis 3.5.1
code like this :
result in console like this: