HGETALL on 2,7 million recrods (9 fields) took 40 seconds

Grokzen / redis-py-cluster

Python cluster client for the official redis cluster. Redis 3.0+.

https://redis-py-cluster.readthedocs.io/

MIT License

1.1k stars 316 forks source link

Closed pnthai88 closed 2 years ago

pnthai88 commented 2 years ago

I'm using pipeline and concurrent.futures.ThreadPoolExecutor . Any idea to get data faster? Whole cluster in Docker,

While doing hgetall on mass, CPU does not consume much

Any idea? Thanks.