hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
10.03k stars 716 forks source link

[FEATURE REQUEST] Allow cancelling task without retrying it #968

Open faleksic opened 1 week ago

faleksic commented 1 week ago

I want to cancel a task without asynq retrying it. I've tried to return asynq.SkipRetry error from the handler when context is canceled, but currently Asynq processor will ignore it and pass context error ctx.Err() to the handleFailedMessage function and my error is ignored.

I wish there is a way to call inspector.CancelProcessing(taskID) and skip retrying that specific task. I am trying to implement a feature where a client can give up on running some task and simply cancel it.

One idea to bypass this issue is to set MaxRetry to 0 and that way it wouldn't be retried, but then the task wouldn't be retried in any other case. How I currently bypassed this issue is that keep my own version of Cancellations and before running the task I create a different context instead of passing the one given to me by Asynq. That way, I am able to handle the cancel event and return asynq.SkipRetry to not retry it again. Asynq is not aware that the task is canceled

kamikazechaser commented 1 week ago

This would be a good feature to have. One way is to handle the context.Cancelled error case and archive the task. The problem is that would be a breaking change because most of the users are used to the fact that any job that has its context cancelled will be retried (current behavior).

We could add an option of archiveOnCancel (bool) to the server which would then archive the task upon context cancellation or if false, retry it (default).

An extra good to have feature would also be to access the Cancellations map directly from the client so that you don't have to cancel through Redis pub/sub which is unreliable.

Also btw, in theory, you could register a custom error handler and archive the task with the inspector to achieve the same effect. Maybe i'll test it and document it in the wiki.

Related:

kamikazechaser commented 1 week ago

Could you try something like this and pass it to the server Config ErrorHandler and report back.

type CancellationsHandler struct {
    inspector *asynq.Inspector
}

func (h *CancellationsHandler) HandleError(ctx context.Context, t *asyq.Task, err error) {
    if errors.Is(err, context.Cancelled) {
          return h.inspector.Archive(...)
    }
}
faleksic commented 1 week ago

Just tried it, unfortunately it doesn't work. I get the following error cannot archive task in active state. use CancelProcessing instead.