Queue Retry for failed jobs

[kolaente]

Description (*)

We're using the magento queuing system (with the mysql provider) to send orders for further processing to a third party system. This external system sometimes fails to process the order. In those cases, we would like to retry sending the order to the external system from magento.

We could just re-queue it, which would retry the push pretty much immediately. However, that solution has it's own issues:

1. If the external system has issues for a longer period, we would try to push orders often which doesn't really help when the system is doing maintenance or something like that.
2. We could run into the "thundering herd" problem where a lot of failed jobs would be retried at the same time, maybe causing the external system to get down again because it can't handle the load.
3. Without further logic in our consumer, we can't retry a failed job only a few times, resulting in failed jobs being retried infinitely (this may or may not be desired, in our case it's not).

This may or may not be a problem only with the mysql provider, we just don't have any experience with the other ones.

Expected behavior (*)

We would like a solution which provides an easy way to retry a failed job a) A configurable amount of times before finally giving up b) After waiting a certain period to avoid the thundering herd problem (sometimes known as "exponentional backoff")

Benefits

Failing jobs are an issue which will always happen. Providing a clean solution for retrying them from within magento would avoid everyone having to cook up their own solution.

Additional information

This has been partly discussed on stack exchange.

Laravel has solved this problem quite nicely: https://laravel.com/docs/8.x/events#handling-failed-jobs

[m2-assistant[bot]]

Hi @kolaente. Thank you for your report. To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

  :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

:movie_camera: You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 14 days if no further activity occurs. Is this issue still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? Thank you for your contributions!

[kolaente]

Yes, still relevant.

[crippsy14]

Bumping this as still relevant, cheers

[onlinebizsoft]

@crippsy14 @kolaente maybe this is something you need? https://github.com/run-as-root/magento2-message-queue-retry