search

antirez / disque

Disque is a distributed message broker
BSD 3-Clause "New" or "Revised" License
8.01k stars 538 forks source link

How to realize such scenario? #142

Closed win32nipuh closed 8 years ago

win32nipuh commented 8 years ago

Simple example, given: 2 applications processing jobs 1 disque server

app1 gets the next job Job1 to process and it can take 10 minutes

  1. How to avoid app2 gets the same Job1 to process?
  2. app1 crashed during the processing. How to return this Job1 back to queue for processing by app2?
  3. disque server crashed and then restarted. What will be with queue and jobs?

Thanks

canardleteer commented 8 years ago

1) Your best bet is probably the WORKING command. 2) The NACK command, and making sure that app1 "crashes" in a healthy enough way to be able to send one. 3) Enable disk persistence.

Regarding #1 & #2, from the README: "Disque implementation of at least once delivery semantics is designed in order to avoid multiple delivery during certain classes of failures. It is not able to guarantee that no multiple deliveries will occur."

canardleteer commented 8 years ago

Also, GETJOB WITHCOUNTERS and SHOW may be helpful in making a decision about accepting a job or not.

antirez commented 8 years ago

About '2', note that even without NACK the job will be delivered again if not acknowledged.