manast
commented
2 days ago I am confused about this request. You can configure the retries as you see fit, you can have your own custom retry strategy, so not sure what is lacking in BullMQ. Furthermore, BullMQ is indeed storing all the errors and stack traces in the job, so you can track the original error and all the other errors raised by that job... So please provide more information as I am not able to understand what you mean.
Is your feature request related to a problem? Please describe.
Currently, BullMQ lacks flexible control over task validity periods, especially for HTTP request-type tasks. For instance, with certain HTTP request tasks, retrying after 30 seconds of no response often becomes meaningless. We need the ability to set shorter validity periods (e.g., 20 seconds) for such tasks, after which retries should cease. Moreover, our current approach marks timed-out tasks as failed, but this overwrites the original error information, preventing us from tracking the root cause of task failures.
Describe the solution you'd like Allow setting custom validity periods for individual tasks or task types. Enable normal retry mechanisms within the validity period. Automatically stop retrying after the validity period expires, while preserving the original error information. Provide a mechanism to track historical error information for tasks, including errors generated during intermediate retry attempts.
Additional context This feature is particularly crucial for handling time-sensitive tasks (e.g., real-time communication, payment processing). It would help us better manage system resources by avoiding wasted time and computational power on tasks that have lost relevance. Additionally, maintaining complete error histories would significantly enhance our debugging and problem diagnosis capabilities.