Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
580 stars 775 forks source link

Negative Scheduled Message Count #653

Closed fil-at-werma closed 9 months ago

fil-at-werma commented 1 year ago

Description

During research I found this closed Issue, that should have fixed this: https://github.com/Azure/azure-service-bus/issues/254

But nontheless, we are encountering a Scheduled Message Count of "-1". It never goes below -1 (as far as we could see).

UPDATE 2023-05-26 : Meanwhile, we did have observed values below -1.

What we do is schedule a message for delayed delivery:

var sequenceNumber = await _serviceBusSender.ScheduleMessageAsync(messageToSchedule, scheduledEnqueueDateTime, cancellationToken);

Where the delay is < 24h and > 0.

If a state in the system changes before that delay is up, we cancel the message by their sequenceNumber:

await _serviceBusSender.CancelScheduledMessageAsync(sequenceNumber, cancellationToken);

Actual Behavior

"Scheduled Message Count" shows "-1" Messages.

grafik

Expected Behavior

"Scheduled Message Count" shows at least "0" Messages.

We are on .Net 6.0, using Azure.Messaging.ServiceBus 7.10.0 Pricing Tier is Standard

EldertGrootenboer commented 1 year ago

We have opened an investigation task for this in our backlog, and will update this issue when we have more information.

perfectphase commented 1 year ago

Just to add, the issue is still there and not limited to showing -1

image

For me this happened when we used the tool Queue Explorer to move schedule messages back to the queue at the same time azure tired to move them back.

fil-at-werma commented 1 year ago

We, too have observed values below -1 since.

JSCProjects commented 1 year ago

This is also happening within our subscriptions. Our sagas depend on scheduled messages and our ops team is complaining that they can't rely on this number.

anth12 commented 10 months ago

I'm seeing this when cancelling scheduled messages.

When a message is scheduled and once the scheduled message is cancelled, the scheduledMessageCount is accurate. However, when cancelling the message expiration date has passed, the scheduledMessageCount goes to a negative value with the cancelled message becoming active.

EldertGrootenboer commented 9 months ago

We have recently released a fix which should address this. If this does happen again in the future, please open a support case, as that will provide us with the information needed to troubleshoot this further.

jrgcubano commented 4 months ago

@EldertGrootenboer Hello. Where is the fix? Client library or do you mean on Azure side?

I'm working on a use case where some events triggered by a system, scheduled a message to later heartbeat an external service... This implies that several events could be competing, so every time we handle an event, we try to cancel any already scheduled message, winning always the latest event. And, due to this bug, the schedule is not cancelled properly and the active message is trigger.

EldertGrootenboer commented 4 months ago

This has been implemented on the service side. If you still see this behavior, please open a support ticket, with all the details about your namespace, so this can be investigated further.

JayPatel1992 commented 2 months ago

This is still happening within our subscriptions. Seeing -1 in the Scheduled Messages.