redis-py and rq version

[eboileau]

Currently, we had to pin versions of redis-py (2.10.6) and rq (0.12). redis-py 3.0 has backwards incompatible changes that breaks the program.

redis.exceptions.DataError: Invalid input of type: 'bool'. Convert to a bytes, string, int or float first.

We should aim to fix this in the near future, and also possibly think about integration with Flask (Redis Queue integration for Flask applications RQ/RQ2).

I suspect these older versions could be the cause of such unstable behaviour: Suddenly, I experience issues with rqworkers that are unlikely to be related to changes in the code (no changes affecting rq were made). With redis-server & rqworker [--path], queues are created without workers, and queries hang forever.

redis-cli smembers rq:workers
(empty list or set)

but e.g.

rqinfo
default      |██ 2
1 queues, 2 jobs total
0 workers, 1 queues

If I start redis-server first, then rqworker after, then this is apparently fine... I have no explanation for this behaviour. This is currently reproducible. I have experienced other issues that are not reproducible, and likely related to redis/redis-py and/or rq interaction. Venv rmap.

[tjakobi]

For the sake of completeness: the redis-server start procedure will not work on the external server, since here the server instance is already started by root.