search

eclipse-paho / paho.mqtt.java

Eclipse Paho Java MQTT client library. Paho is an Eclipse IoT project.
https://eclipse.org/paho
Other
2.15k stars 895 forks source link

Timed out as no activity #716

Open 2422006980 opened 5 years ago

2422006980 commented 5 years ago

My program subscribed to a topic, which received a lot of subscription information. After running for a period of time, the program would always report the following errors.

ERROR 2019-11-29 14:17:42 - 56480046-7dbd-4500-b074-b9e729a6d26c: Timed out as no activity, keepAlive=60,000,000,000 lastOutboundActivity=73,870,534,312,909,049 lastInboundActivity=73,870,474,311,957,613 time=73,870,594,312,986,154 lastPing=73,870,534,312,910,698 Timed out waiting for a response from the server (32000) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:31) at org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity(ClientState.java:741) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:816) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:802) at org.eclipse.paho.client.mqttv3.TimerPingSender$PingTask.run(TimerPingSender.java:79) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Timed out waiting for a response from the server (32000) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:31) at org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity(ClientState.java:741) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:816) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:802) at org.eclipse.paho.client.mqttv3.TimerPingSender$PingTask.run(TimerPingSender.java:79) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Timed out waiting for a response from the server (32000) WARN 2019-11-29 14:17:42 - Connection is broken：Timed out waiting for a response from the server， at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:31) at org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity(ClientState.java:741) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:816)INFO 2019-11-29 14:17:42 - Connecting to broker: tcp://192.168.1.124:1883

    at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:802)
    at org.eclipse.paho.client.mqttv3.TimerPingSender$PingTask.run(TimerPingSender.java:79)
    at java.util.TimerThread.mainLoop(Timer.java:555)
    at java.util.TimerThread.run(Timer.java:505)
deadstar2009 commented 4 years ago

Confirm. A bug with this problem exists and appears completely unpredictable. It doesn't matter which host you connect to with the broker. I have a problem when connecting via loopback (127.0.0.1), and there is nothing in the broker's logs that could suggest the cause of the problem. It would seem that there may be a problem when connecting to yourself... Today i have new error:

Timed out waiting for a response from the server (32000) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:31) at org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity(ClientState.java:741) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:816) at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:802) at org.eclipse.paho.client.mqttv3.TimerPingSender$PingTask.run(TimerPingSender.java:79) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505)

rdasgupt commented 4 years ago

@deadstar2009 If you have paho client log, could you upload it?

rdasgupt commented 4 years ago

@2422006980 @deadstar2009 In your client logs, do you see log messages like: ... FINE 20-02-19 08:26:55.0072 nternal.ClientState notifySent 13 IBMIoTClientSub: ping sent. pingOutstanding: 1 FINE 20-02-19 08:26:55.0072 nternal.ClientState get 13 IBMIoTClientSub: wait for new work or for space in the inflight window .... .... SEVERE 20-02-19 08:27:55.0075 nternal.ClientState checkForActivity 15 IBMIoTClientSub: Timed out as no activity, keepAlive=60,000,000,000 lastOutboundActivity=490,258,899,517,044 lastInboundActivity=490,198,894,436,119 time=490,318,902,483,410 lastPing=490,258,899,932,033 FINE 20-02-19 08:27:55.0076 nternal.ClientComms handleRunException 15 IBMIoTClientSub: exception Throwable occurred: Timed out waiting for a response from the server (32000) .....

Basically look for two things:

If you have these logs, paste the log messages. If possible upload the log file.

rldaulton commented 2 years ago

Is this still a valid issue?

fbertini commented 2 years ago

Hello,

I see this issue is still present on version 1.2.5 and it is happening regularly.

jone-code commented 1 year ago

+1

lorivalmatias commented 1 year ago

+1

uiTravis commented 9 months ago

The root cause is delta variable at the checkForActivity function which in ClientState.java file. I declare a new pingDelta variable and set the value to delta * 1000 to expand gap for broker ping response. It works.

Arvin-Li1 commented 6 months ago

I also meet this issue, and i saw someone who provided a workable way to fix this issue, is that the performance of the server which accept message from mqtt is so low that cannot process messages, so the tcp buffer of server is full, and cannot receive new message from mqtt. inproving server performance may help to fix this issue, like multi-threading, aysnc... but I havent tried it, hope someone who can attempt

tayweekiat commented 2 weeks ago

hi... want to add that I experienced this issue too on our dev server...

just for some context... our Java application talks to MySQL 8 database as well as send message to MQTT Server using the Paho MQTT Java client (version 1.2.5)... we have not encountered this issue before until recently we wanted to test out creating database index on a very big table (about 3mil rows) to see if the indexing would impact the application... that is when we started seeing the MQTT errors appearing...

our MQTT server (running on ActiveMQ) is still working fine as I could still connect to it and send/receive message using an independent MQTT client... however the database table is being locked up due to the indexing...

we managed to repeat this error even on a new ActiveMQ server instance... which stumped me on why would database table getting locked up cause the Paho library client to throw this error since the ActiveMQ has nothing to do with our MySQL database...?

Hope that this info is helping to understand what is causing this bug to happen...