search

marvinroger / async-mqtt-client

📶 An Arduino for ESP8266 asynchronous MQTT client implementation
MIT License
834 stars 266 forks source link

Unable to subscribe to last topics #212

Closed joelgamal closed 3 years ago

joelgamal commented 3 years ago

I am publishing 6 quite large payloads on 6 different topics then trying to subscribe to some topics. It subscribes successfully to first topics but fails to last ones. I am using esp8266

luebbe commented 3 years ago

Do your payloads arrive on the MQTT server? If they don't, you won't receive anything.

kleini commented 3 years ago

If I understand the issues with this library correctly, it is not capable to send more data, than what fits into TCP_MSS (1460) if the send packages are not acknowledged. I found somewhere the fix by using an AsyncTCPBuffer for the send queue, which solves at least my issues with to much data to be sent: https://github.com/kleini/async-mqtt-client/commit/f1b42054815ad82fed4519bce7febb7f1601560f

bertmelis commented 3 years ago

If I understand the issues with this library correctly, it is not capable to send more data, than what fits into TCP_MSS (1460) if the send packages are not acknowledged. I found somewhere the fix by using an AsyncTCPBuffer for the send queue, which solves at least my issues with to much data to be sent: kleini@f1b4205

Indeed, define "large payload". I'm working on a solution but it needs implementation and testing.

Now, subscribing should not be a problem. What is the return value when calling subscribe?

joelgamal commented 3 years ago

large payload: about 400 char each I guess as @kleini said it is not capable of sending more data /do any function(subscribe). when I reduced payload size, it subscribes successfully to all topics subscribe returns 0

bertmelis commented 3 years ago

When it returns 0 it means the subscribe call was not successful. You should try again later. When the TCP buffer is full and is waiting for an ACK to purge it's buffer, it does not accept more data. Hence, the subscribe call fails.

joelgamal commented 3 years ago

Can I increase buffer size? Also what do you mean by waiting for ACK to purge? Can I purge it manually? I am using QoS 0

bertmelis commented 3 years ago

No you can't. It's defined by LWIP. The broker acknowledges the data. It's not something the sending party does.

You can wait for a solution (I'm working on one but no deadline, sorry. Can take weeks) or you can use @kleini 's fork.

proddy commented 3 years ago

@joelgamal 400 bytes is not very big. I'm sending 10's of messages every few seconds with payloads much larger. Just put some delay(x) after a publish so it doesn't flood the TCP pipe. Also compiling using other lwip variants (like the V2 higher bandwidth, or in pio -D PIO_FRAMEWORK_ARDUINO_LWIP2_HIGHER_BANDWIDTH) will double the buffer size. But I'm not an expert here...

dm2302git commented 3 years ago

If I understand the issues with this library correctly, it is not capable to send more data, than what fits into TCP_MSS (1460) if the send packages are not acknowledged. I found somewhere the fix by using an AsyncTCPBuffer for the send queue, which solves at least my issues with to much data to be sent: kleini@f1b4205

I've tried it with the workaround. But then only one part of the message is published. Is there any limitation ?

kleini commented 3 years ago

I've tried it with the workaround. But then only one part of the message is published. Is there any limitation ?

Yes, 1500 bytes for the whole package: https://github.com/kleini/async-mqtt-client/blob/f1b42054815ad82fed4519bce7febb7f1601560f/src/AsyncMqttClient.cpp#L878

joelgamal commented 3 years ago

okay thank you so much, waiting for the update though

kimamov commented 3 years ago

If I understand the issues with this library correctly, it is not capable to send more data, than what fits into TCP_MSS (1460) if the send packages are not acknowledged. I found somewhere the fix by using an AsyncTCPBuffer for the send queue, which solves at least my issues with to much data to be sent: kleini@f1b4205

Could you please describe what your issue was? I am trying to use your version for sending around 20 files inside a SPIFFS folder with while(dir.next()). Usually it would send around 6 messages before stopping but with your version it only sends one. So was your use case different? Thank you!

kleini commented 3 years ago

@kantimam My issue was, that with using https://github.com/homieiot/homie-esp8266 and too much Homienodes I had too much small MQTT messages for advertising these nodes that filled in total TCP_MSS and further messages could not be sent. This results then in not advertised nodes. My small fix solves that issue by putting all these small messages in an addittional buffer on top of TCP_MSS and TCP_MSS is not used that much any more. In your case the problem might be, that a single MQTT message might only have a size of 1500 bytes. If you need larger messages, we need to fix that somehow. Furthermore the problem can be, that the ESPAsyncTCP buffer runs out of memory, although I did not check this in detail.

kimamov commented 3 years ago

@kantimam My issue was, that with using https://github.com/homieiot/homie-esp8266 and too much Homienodes I had too much small MQTT messages for advertising these nodes that filled in total TCP_MSS and further messages could not be sent. This results then in not advertised nodes. My small fix solves that issue by putting all these small messages in an addittional buffer on top of TCP_MSS and TCP_MSS is not used that much any more. In your case the problem might be, that a single MQTT message might only have a size of 1500 bytes. If you need larger messages, we need to fix that somehow. Furthermore the problem can be, that the ESPAsyncTCP buffer runs out of memory, although I did not check this in detail.

thanks alot. Looks like using qos1 instead of 0 when sending inside a loop fixed my problem :)

bertmelis commented 3 years ago

This should have been solved in the develop branch.