redis queue is not support reconnect redis server

[chenzi2015]

Hi Everyone

I use redis as persistence queue, and have a problem that redis queue is not support reconnect redis server.

• If gearmand is running , The network is disconnect or redis server restart.

This can lead to got error:

failed to insert 'gear-reverse-c9984240-b20d-11e6-9924-080027596944' into redis _hiredis_add(QUEUE_ERROR)

[p-alik]

I guess it's expected behaviour. Gearmand puts jobs into persistence layer only on shutdown.

[chenzi2015]

@p-alik I think it's not expected behaviour, beacase if network is disconnect or redis server restart will to cause the redisContext invalid. And client can't puts jobs into Gearmand server.

[SpamapS]

It would be a nice feature to keep trying. It's not reallya bug, so I'll mark it wishlist.

However, I don't know how to say this more strongly: The gearmand queue modules are the least scalable way to do persistent messaging. If you want to scale this out, I would recommend running workers on each redis server that inject the data into Redis locally, acknowledge the job when it's safely in redis, and then re-submit the job as synchronous jobs for the real workers.

[lkebin]

I have same problem.

I did setup Redis as persistence queue. And I think the jobs will be put into Redis when Gearman server shutdown.

But when Gearman server is running, I restart my Redis server, then the client submit doBackground will be failed.

PHP Warning:  GearmanClient::doBackground(): _client_run_tasks(GEARMAN_QUEUE_ERROR) QUEUE_ERROR:QUEUE_ERROR -> libgearman/client.cc:1594: pid(4931) in /home/lkebin/client.php on line 6

The Gearman server log have messages:

 DEBUG 2017-07-12 13:50:41.022612 [  proc ] PACKET COMMAND: GEARMAN_SUBMIT_JOB_BG -> libgearman-server/server.cc:119
  DEBUG 2017-07-12 13:50:41.022625 [  proc ] Received submission, function:gid unique:17d3feee-6709-11e7-8553-00163e0e0951 with 2 arguments -> libgearman-server/server.cc:248
  DEBUG 2017-07-12 13:50:41.022633 [  proc ] Comparing queue 5 to limit 0 for priority 1 -> libgearman-server/job.cc:175
  DEBUG 2017-07-12 13:50:41.022643 [  proc ] JOB H:srv-zanadu-bj-test1:4238 :2891878068 -> libgearman-server/job.cc:240
  DEBUG 2017-07-12 13:50:41.022648 [  proc ] hires add: 17d3feee-6709-11e7-8553-00163e0e0951 -> libgearman-server/plugins/queue/redis/queue.cc:231
  DEBUG 2017-07-12 13:50:41.022655 [  proc ] hires key: 50 -> libgearman-server/plugins/queue/redis/queue.cc:237
  DEBUG 2017-07-12 13:50:41.022668 [  proc ] got reply -> libgearman-server/plugins/queue/redis/queue.cc:241
  ERROR 2017-07-12 13:50:41.022671 [  proc ] failed to insert '_gear_-gid-17d3feee-6709-11e7-8553-00163e0e0951' into redis _hiredis_add(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:245
  ERROR 2017-07-12 13:50:41.022676 [  proc ] gearman_server_job_add gearman_server_run_command(QUEUE_ERROR) -> libgearman-server/server.cc:301

Gearman server never been shutdown, but it do something with Redis, and failed.