antirez / disque

Disque is a distributed message broker
BSD 3-Clause "New" or "Revised" License
8.01k stars 537 forks source link

RFE: WAITJOB, a blocking operation until the job is ACKed #168

Open VojtechVitek opened 8 years ago

VojtechVitek commented 8 years ago

WAITJOB id timeout - blocks until the job is ACKed or until the timeout is reached

Use case: Multiple consumers waiting on a single job to be done, so they can move on..

lovelle commented 8 years ago

+1 on this

Revisor commented 8 years ago

Out of curiosity, why would multiple consumers be waiting for one job to be processed?

VojtechVitek commented 8 years ago

@Revisor Let's say you have a Travis job and multiple people waiting on the result (UI on the Pull Requests).

lovelle commented 8 years ago

@VojtechVitek Sorry if I'm wrong or if is not what you mean, but you can do GETJOB until timeout is reached.

127.0.0.1:7711> GETJOB TIMEOUT 10000 FROM test
(nil)
(10.07s)
VojtechVitek commented 8 years ago

@lovelle GETJOB gives you next available job from the queue and put it into the "active" state.

WAITJOB would wait on a specific job until it's "acked".

lovelle commented 8 years ago

@VojtechVitek ook, now I get it, sorry for confusion :+1:

dhagrow commented 8 years ago

Tacking on to this: in addition to WAITJOB I would like QWAIT as well.

For my use case I queue many jobs (thousands, even millions) in a batch and I need to know when the batch completes. Not just when they're all "active" which I can get from QLEN, but when they have all actually been "acked". Currently there doesn't appear to be any way to do this at all without scanning all jobs directly for their state.

WAITJOB would not be sufficient for this.

jiusanzhou commented 7 years ago

Is there any way to get the job which is acked with the job ID?

kaspar030 commented 7 years ago

I've build a CI around disque, my solution was to let the job, before completion, send a "I'm done" job to a previously defined status queue.