Azure / azure-functions-servicebus-extension

Service Bus extension for Azure Functions
MIT License
65 stars 35 forks source link

ServiceBusTrigger delayed up to 12 hours before executing #116

Closed timgabrhel closed 2 years ago

timgabrhel commented 3 years ago
  <ItemGroup>
    <PackageReference Include="AutoMapper" Version="9.0.0" />
    <PackageReference Include="AutoMapper.Extensions.Microsoft.DependencyInjection" Version="7.0.0" />
    <PackageReference Include="Microsoft.AspNetCore.Hosting" Version="2.2.7" />
    <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.0.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.ServiceBus" Version="4.1.1" />
    <PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="3.1.3" />
    <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="3.1.3" />
    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="3.1.3" />
    <PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="3.1.3" />
    <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="3.1.3" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.6" />
    <PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
  </ItemGroup>

My primary ServiceBusTrigger function is not firing when messages are available in the queue. This function has been running in production for 5 months. Starting last weekend, we saw the number of queue messages climbing and the number of function executions dropping. To be clear, the function works as expected, then simply stops executing.

In my Startup.cs, I'm configuring the auto-completion of messages.

builder.Services.Configure(delegate(ServiceBusOptions options)
{
    options.MessageHandlerOptions.AutoComplete = false;
});

The function uses a ServiceBusTrigger. function signature

App Insights function activity shows normal processing, then complete silence. The large blocks to the right are where the function fires for the bulk of messages at once. app insights log

The volume of messages within this specific queue correlates to the app insights log. Processing as normal, then the number of notifications slowly grows until they are all processed at once.

queue depth

alrod commented 2 years ago

Closing the old issue. @timgabrhel , next time please open a new support case.