Azure / azure-functions-dotnet-worker

Azure Functions out-of-process .NET language worker
MIT License
412 stars 176 forks source link

Is it possible to disable automatic abandoning of service bus messages? #2536

Closed havarnov closed 2 months ago

havarnov commented 2 months ago

A "vanilla" ServiceBusTrigger will complete the message on success and abandon a message on any Exception. If I set the the AutoCompleteMessages = false messages will not be automatically completed on success, but still be automatically abandoned on any Exception.

In some cases it would be nice let the message not be automatically abandon, e.g. in a scenario if there's a transient error and I want the message to not be retried immediately. Something like:

[Function(nameof(ServiceBusPlayground))]
public async Task ServiceBusPlayground(
    [ServiceBusTrigger(
        "testtopic",
        "testsubscription",
        AutoCompleteMessages = false,
        AutoAbandonMessages = false,
        Connection = "AzureServiceBus")]
    ServiceBusReceivedMessage message,
    ServiceBusMessageActions messageActions,
    CancellationToken hostCancellationToken = default)
{
    try
    {
        await SomethingThatCanFail();

        await messageActions.CompleteMessageAsync(message, hostCancellationToken);
    }
    catch (TransientException e)
    {
        // renew to make sure the message stays locked for as long as possible.
        await messageActions.RenewMessageLockAsync(message, hostCancellationToken);
    }
    catch (Exception e)
    {
        await messageActions.AbandonMessageAsync(message, cancellationToken: hostCancellationToken);
        throw;
    }
}

As far as I can tell, this is not currently possible? What is the recommended pattern in this scenario? Should I create a separate queue for handling retries (then I'm not able to use the built-in Max delivery count functionality in service bus)?

PS: Is the "worker" code open source (or source available)? It would be nice to se how service bus triggers (and others) are handled.

SeanFeldman commented 2 months ago

Auto-completion of a message is a built-in feature of the Service Bus SDK (via ServiceBusMessageProcessor) and is not unique to Azure Functions. Azure Functions trigger exposes this with the AutoCompleteMessages option of the Function attribute. Abandonment, on the other hand, is very scenario-specific. So what you have with try/catch looks suitable for your scenario. If you have all of your functions that require the same behaviour, middleware can help.

havarnov commented 2 months ago

I don't think I expressed myself clear enough. My try..catch example above is not possible with the automatic abandonment as far as I can tell.

So my question is, is it possible to control abandonment of messages (with the azure function trigger) and if not, what is the recommended pattern to handle delayed retries with service bus and azure function.

SeanFeldman commented 2 months ago

So my question is, is it possible to control abandonment of messages (with the azure function trigger) and if not, what is the recommended pattern to handle delayed retries with service bus and azure function.

Now I see what you are after. You're looking to implement delayed retries with a Service Bus trigger. In that case, abandoning a message won't help you because the abandoned message will be picked up shortly after or immediately. If you want to abandon with a delay, I suggest you upvote this feature request.

Azure Functions doesn't provide the ability to delay message retries. If you need that functionality today, you must implement it yourself. I wrote a blog post a while ago that might give you an idea for a direction. Note that since then, the SDK has added support for ServiceBusMessageActions which makes the implementation outlined in the post somewhat simpler.

havarnov commented 2 months ago

In that case, abandoning a message won't help you because the abandoned message will be picked up shortly after or immediately.

I know. What I suggested in my example above is to not abandon the message in some cases, making the message invisible due the lock still being held until the lock expires.

Anyways, thanks for the info and the blog post!