The scrapy-cluster support redis cluster?

[madisonb]

Scrapy Cluster's core is based around redis-py, which does not support cluster mode. In practice a single Redis instance is fast enough to handle almost all large scale operations.

If this answer's your question please close the ticket

[gongshaojie12]

Thanks

[mannmann2]

Is there any way to set up a cluster though? I'm looking for a way to prevent crawling failure if the redis node goes down.