Azure / azure-service-bus-java

☁️ Java client library for Azure Service Bus
https://azure.microsoft.com/services/service-bus
MIT License
60 stars 59 forks source link

TimeoutException during completion of requests (v. 1.2.11) #344

Closed SerJey closed 5 years ago

SerJey commented 5 years ago

Hello,

We faced with some issue. It is similar to #312, but I'm not sure that there is the same root cause.

TimeoutException occures periodically during receiving of messages.

Message cannot be processed due processing failure COMPLETE-Request timed out.
com.microsoft.azure.servicebus.primitives.TimeoutException: Request timed out.
        at com.microsoft.azure.servicebus.primitives.CoreMessageReceiver$1.run(CoreMessageReceiver.java:162) ~[azure-servicebus-1.2.11.jar!/:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) ~[?:?]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) ~[?:?]
        at java.lang.Thread.run(Thread.java:844) ~[?:?]

We use version 1.2.11, but issue is reproducing in previous versions too.

Our config:

The issue is more reproducible in case when we run 2 or more instances of our application.

yvgopal commented 5 years ago

A timeout could be either a client side issue or service side issue. Could you identify any pattern when these timeouts occur? That will help me identify if it is a client issue or service issue. Try with a different namespace and see if you can reproduce it.

mrdfuse commented 5 years ago

There is not a real pattern to see, we send health messages every few 30 seconds or so, sometimes we get this exception, seemingly randomly. This on a system with no other activity. We also see this happening on different namespaces, both Premium and Standard.

yvgopal commented 5 years ago

One possible indicator: If a timeout occurs and continues to occur, then it is likely a client side issue. If a timeout occurs, but subsequent operations succed, it is likely a timeout from the service. Now tell me what you think it is.

mrdfuse commented 5 years ago

Probably service then. I also noticed that about 10 days ago the exceptions stopped appearing on all our environments, without us having deployed new version of our client applications.