In our usage of backburner we have quite a few expected error cases, especially when interacting with storage systems that use optimistic concurrency control. If two jobs are competing to update the same resource, one of them will fail.
The only means (that I know of) of retrying a job is to raise an exception or to enqueue a new one (which is undesirable as it will lead to a bunch of serialization and IO). Raising an exception each time is leading to a lot of noise in our error logs
I can think of 2 ways to solve this issue:
Create a specific Exception that is handled explicitly in worker.rb and simply retries the job (using the current increment and delay logic).
Return a symbol from the processor to indicate if a job should be retried ie :retry.
@contentfree @nesquena what do you think?
eltone
commented
5 years ago
In our usage of backburner we have quite a few expected error cases, especially when interacting with storage systems that use optimistic concurrency control. If two jobs are competing to update the same resource, one of them will fail.
The only means (that I know of) of retrying a job is to raise an exception or to enqueue a new one (which is undesirable as it will lead to a bunch of serialization and IO). Raising an exception each time is leading to a lot of noise in our error logs
I can think of 2 ways to solve this issue:
:retry. @contentfree @nesquena what do you think?