Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
585 stars 781 forks source link

AutoRenewTimeout is ignored #24

Closed JohnHanna86 closed 7 years ago

JohnHanna86 commented 7 years ago

I am using service bus queue; I have this setting for the OnMessageOptions

var onMessageOptions = new OnMessageOptions
            {
                MaxConcurrentCalls = 10,
                AutoRenewTimeout = TimeSpan.FromMinutes(100),
                AutoComplete = false
            };

and i set the Lock Duration for the queue to 1 min, so i assume that the lock will not expire untill either i complete the message or the lock is renewed more 100 times. is that correct?

What actually happens the lock is expired after 1 minute, and I get the message delivered again while the old message is still in processing!

SeanFeldman commented 7 years ago

i assume that the lock will not expire untill either i complete the message or the lock is renewed more 100 times. is that correct?

No, you're setting renew timeout up to 100 minutes.

Message lock renewal is a call to the broker. If that call fails, then, well, you won't get a lock token and message will be available to other competing consumers for processing. In that case, your processors should be idempotent.

JohnHanna86 commented 7 years ago

This still does not answer my question, my main concern is that the message gets picked up by another receiver before the AutoRenewTimeout is up, here is my code:

class ServiceBusQueueExample
    {

        private QueueClient queueClient;
        private readonly ManualResetEvent completedEvent = new ManualResetEvent(false);

        public void Run()
        {
            string connectionString = "XXX";
            string queueName = "queueName";
            queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName);

            var onMessageOptions = new OnMessageOptions
            {
                MaxConcurrentCalls = 10,
                AutoRenewTimeout = TimeSpan.FromMinutes(100),
                AutoComplete = false
            };

            queueClient.OnMessageAsync(
                async receivedMessage =>
                {
                    Console.WriteLine($"received message, Id:{receivedMessage.MessageId} - DeliveryCount:{receivedMessage.DeliveryCount}");
                    //simulates the message processing
                    await Task.Delay(TimeSpan.FromMinutes(40));
                    Console.WriteLine("finished processing");
                    await receivedMessage.CompleteAsync();
                    Console.WriteLine("message is completed");
                },
                onMessageOptions);

            completedEvent.WaitOne();

        }
    }

In my code, before the message processing is finished, I find it received again and gain by different receivers, I assume this is not the intended behavior.

SeanFeldman commented 7 years ago

Added perf counters, and as you can see things are not smooth. image

When you ask ASB client to auto renew message lock, it will just issue a periodic request to the broker. Those requests can fail. Sometimes it's the connectivity or some messaging exception that the clients recovers from quickly, but that time is enough to lose the lock token. The longer total processing time is, the more chances one of the renewal requests will fail. This is expected. I would suggest either lower the processing time or review the approach.