search

davechallis / ocypod

Ocypod is a Redis-backed service for orchestrating background jobs. Clients/workers can be written in any language, using HTTP/JSON to queue/fetch jobs, store results, etc.
Apache License 2.0
193 stars 13 forks source link

Use PATCH instead of GET to dequeue a job #14

Open rhuijts opened 3 years ago

rhuijts commented 3 years ago

Hi, I love the API but I have a suggestion for the GET /queue/{queue_name}/job endpoint. The documentation says: "When a client gets a job in this way, the job is marked as running, and is removed from the queue." GET requests should be safe (not modify the resource) and idempotent (multiple request have the same effect as a single one). Perhaps use the PATCH method because the worker wants to change the created status to running. This gives workers two options:

  1. Use PATCH /queue/{queue_name}/job to mark the next job on the queue as running and retrieve the id and input in one atomic operation.
  2. Use GET /queue/{queue_name}/job to only "peek" at the next available job, to decide if they want it or not, and then PATCH /job/{job_id} to mark it as running. If another worker already took it, the response code is 409 (Conflict).

Does that make sense? Thanks anyway, this is a nice job queue server!

davechallis commented 3 years ago

Thanks - I definitely agree here. It's a tricky one - we discussed this when working on the initial API, and went back and forth between using GET (a simpler but less RESTful approach), and PATCH/POST/DELETE (more RESTful, but slightly more complex to use).

In the end, GET was chosen as this matched the behaviour of other task queue systems we were considering.

What I'll probably end up doing (after a bit more design work) is adding new behaviour to use a non-idempotent verb for getting the latest job from the queue, and deprecating the GET endpoint (and eventually removing it at some point).