search

hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
9.42k stars 681 forks source link

[FEATURE REQUEST] Add ability to remove unique tasks on error #629

Open lexuzieel opened 1 year ago

lexuzieel commented 1 year ago

Is your feature request related to a problem? Please describe. In my project I run scheduled tasks with drastically varying execution times, so I schedule them to run every minute and tried to use unique option in order to have "no-overlap" logic. However, since currently unique lock is only released on successful completion, in case my task ends with an error I have a dangling unique task indefinitely (in order to achieve no overlapping I set unique TTL to a large amount - like an hour).

Describe the solution you'd like I would like to have an option to tell that unique lock should be released on task error.

Describe alternatives you've considered Currently the only option I see is to manually remove the task using Inspector at the end of processor handler.

Additional context Coming from Laravel, it has a concept of "without overlapping" job scheduling: https://laravel.com/docs/10.x/queues#preventing-job-overlaps

kamikazechaser commented 1 year ago

Better to use a distributed lock mechanism like https://github.com/bsm/redislock then defer release the lock in the task handler.