antirez / disque

Disque is a distributed message broker
BSD 3-Clause "New" or "Revised" License
8.01k stars 538 forks source link

QLEN inconsistent results #206

Open dolzenko opened 6 years ago

dolzenko commented 6 years ago

Given the cluster of 3 nodes, shouldn't the QLEN return the same results regardless the node being asked? In my tests only original node returns proper queue length, but GETJOB on different node still works properly

$ redis-cli -h eu1-queue-3 -p 7712 addjob testqueue testjob 1000
D-54bdf6f9-2qZjI4HgYKs8vP0ZWSeAANbK-05a1

$ redis-cli -h eu1-queue-1 -p 7712 qlen testqueue
(integer) 0 # WHY?
$ redis-cli -h eu1-queue-2 -p 7712 qlen testqueue
(integer) 0
$ redis-cli -h eu1-queue-3 -p 7712 qlen testqueue
(integer) 1

$ redis-cli -h eu1-queue-1 -p 7712 getjob from testqueue
1) 1) "testqueue"
   2) "D-54bdf6f9-2qZjI4HgYKs8vP0ZWSeAANbK-05a1"
   3) "testjob"
mathieulongtin commented 6 years ago

Try this instead $ redis-cli -h eu1-queue-3 -p 7712 addjob testqueue testjob 1000 replicate 3

dolzenko commented 6 years ago

@mathieulongtin I could, but isn't the default to replicate to all nodes? I would expect QLEN to be consistent with GETJOB :/

mathieulongtin commented 6 years ago

It's not replicated by default. By not specifying replicate, you're asking for one copy. You're saying "I don't need this job to survive a server failure".

dolzenko commented 6 years ago

Sorry for annoyance but README claims replication is by default. Also if jobs are not replicated by default does that mean that clients need to find out cluster size and set replicate param accordingly?

Disque is a synchronously replicated job queue. By default when a new job is added, it is replicated to W nodes before the client gets an acknowledgement about the job being added. W-1 nodes can fail and the message will still be delivered.

dolzenko commented 6 years ago

Ooop, wrong button