Closed kusmierz closed 6 years ago
Seems a sane idea :+1: Haven't looked at the code that much, @basz and @bakura10 might have an opinion about it as well.
I find the idea nice, however this does not apply to SQS :).
SQS keeps track itself of how many times a job was received, and has a feature called "dead letter queue". You can configure to say SQS "hey, after 50 tries, remove the job, or move it to another queue called the dead letter queue for inspecting what happen and why it failed). So I'd be afraid that such a mechanism would enter into conflict.
I'm wondering if other systems such as Beanstalk do not have a similar mechanism. But if they don't, I'm not against the idea, but it should definitely be completely optional and not be activated by default ;).
@bakura10 I was looking into, and can't find any similar mechanism for Beanstalk. Nevertheless, you probably can't find for Doctrine and Redis either :) So SQS is exception here.
BTW - I don't think it would be a conflict (as long as you exactly know where and what limit is set).
I think, you could use "dead letter queue" feature as safety check, just to be sure it has been deleted after a while - but you could also use Slm strategy. What's more - this solution would be independent to Queue implementation - you could change it later to another one, and everything will work exactly the same.
PS
Why SQS' method push()
has delay_seconds
option instead of delay
?
I'll close this one as it more than two years old. Feel free to open new issue if ideas still relevant and / or interested in fixing! :)
As far as I understand, throwing an exception in a job (any
\Exception
for Beanstalk, Redis and SQS or\SlmQueueDoctrine\Job\Exception\ReleasableException
for Doctrine, which btw is different than the rest) will reinsert the job (automatically) for another try. It's good, BUT. What if I have some error(s) in my job's content itself? It would reinsert for infinity...To resolve it, I propose
MaxRecoverableFailuresStrategy
.it will reinsert a job with updated metadata "failure_recoverable_count", and delay it for some time (for greater "failure_recoverable_count" values will be greater delays).
What do you think, @juriansluiman? If it's good idea, I could add delay param options (to adjust it to your needs from config) and do PR.