Rapid Messages will Crash the Client?

[kerti]

Hi! First of all, kudos on your great work.

I've been using your library to subscribe to an MQTT topic, and then relay the message to another Arduino "field unit" via RF links using RadioHead. Here's the sketch if you'd like to take look at it:

https://github.com/kerti/catleya-arduino/blob/master/baseStation/baseStation.ino

I use openHAB to publish messages to MQTT based on what switch I press on my handset. So, all goes well when I test one switch and just leave it alone. But when I mash on the controls and make openHAB spit out a bunch of messages rapidly, eventually the Arduino just goes silent, and I presume it's crashed.

I don't know what happened, maybe you could check it out and see if I crashed the client?

[knolleary]

Hi, I don't see anything obviously wrong with your code.

I've not generally had an issues like this, but then I'm not sure I've ever tried swamping the device with messages.

What have you done to try to debug it?

[kerti]

All I've done is add some code to check my available memory, because my first suspicion was that I somehow used up all memory. This isn't the problem. I haven't tried anything else.

If you have any other ideas I'd be happy to try it out and report back.

[kerti]

Reporting back, I think we simply shouldn't send that many messages at a time. I modified my server to limit itself to a maximum of 1 message per 10 seconds, and it seemed to work fine.

[swell-d]

I had the same problem. After some time my arduino stops to respond. Problem was with my DHCP server. I set static IP on my arduino and my router set the same IP to another device on my network =)

[swell-d]

kerti, how did you limit the server?

[kerti]

swell-d, I use openHAB to send my MQTT messages. What server do you use?

[swell-d]

The same. I use MQTT event bus for outgoing messages.

[kerti]

@swell-d, I simply created an item that I don't show on my UI, and use rules to compare it to the actual items in the UI. An example would make explaining easier:

I have an 8-channel relay attached to an Arduino which I complained about at the top of this thread. I currently have these items:

gRelays_Garage_RELAY01_Device01
gRelays_Garage_RELAY01_Device02
gRelays_Garage_RELAY01_Device03
gRelays_Garage_RELAY01_Device04
gRelays_Garage_RELAY01_Device05
gRelays_Garage_RELAY01_Device06
gRelays_Garage_RELAY01_Device07
gRelays_Garage_RELAY01_Device08

I add another item that I don't show on my UI:

gRelays_Garage_Status

At this point, I don't use any bindings on any items.

I create a rule that looks something like this:

rule "[Update RELAY.01]"
when
    Time cron "10 * * * * ?" // check every 10 seconds
then
    var int statusMask = 0

    if (gRelays_Garage_RELAY01_Device01.state == ON)
        statusMask = statusMask + 1

    if (gRelays_Garage_RELAY01_Device02.state == ON)
        statusMask = statusMask + 2

    if (gRelays_Garage_RELAY01_Device03.state == ON)
        statusMask = statusMask + 4

    if (gRelays_Garage_RELAY01_Device04.state == ON)
        statusMask = statusMask + 8

    if (gRelays_Garage_RELAY01_Device05.state == ON)
        statusMask = statusMask + 16

    if (gRelays_Garage_RELAY01_Device06.state == ON)
        statusMask = statusMask + 32

    if (gRelays_Garage_RELAY01_Device07.state == ON)
        statusMask = statusMask + 64

    if (gRelays_Garage_RELAY01_Device08.state == ON)
        statusMask = statusMask + 128

    if (statusMask.toString() != gRelays_Garage_Status.state.toString())
        sendCommand(gRelays_Garage_Status, statusMask)
end

I can then use the gRelays_Garage_Status to trigger sending the MQTT message, either by binding or by rules. I hope this helps.

[swell-d]

Yes! Thank you

[sjcliffe]

I ran into this problem too but found a workaround using the mosquitto broker.

By sending the messages with a QOS of 1 and setting the mosquitto config variable max_inflight_messages to 1 results in the messages being delivered to the subscriber one at a time - i.e. message 2 will only be sent after the client ACKs message 1.

[JamesM85]

I have also been experiencing this issue and have got around it by using QOS of 1.

In my case just sending two messages consecutively would sometimes cause it to crash. Are there any plans to investigate the issue further? Thanks James

[bensuffolk]

I have just experienced this issue. I have built a module with 8 relays and a command looks like "/VBCM/xxxxxxxx/01/set" = "on" where xxxxxxxx is the serial number and 01 is the channel number.

If I send all 8 channels,one after the other 1 time out of 3 it tends to lock up. Sometimes it resets the sketch, most of time it does not.

I can reproduce this pretty consistently so am going to see if I can debug it.

[bensuffolk]

Ok,

After some fairly extensive testing I find that I have a hardware issue and nothing is wrong with the library. It's all to do with my relay circuit, if I don't have the relays connected there is no problem.

So that's good news for the library, and I need to make some tweaks to my design.

Ben

[cristovao-trevisan]

I had the same problem, but upgrading the library solved it. Check if you are using the last version (2.6 at the time I wrote this).

[MBerka]

Confirming @cristovao-trevisan : I was using the imroy fork that apparently split off around v.1.9, and seeing the same there whenever two messages arrived at once (memory was tight and the client's loop() was not being called very often). Returning to this system (now 2.7) eliminated crashes. There were still occasional disconnects, which went away after restructuring to call the loop constantly.

Recommend closure.

[sblantipodi]

same problem here on ESP32, same code works well on ESP8266 but it crash on ESP32. if I send 5 or 6 message per second ESP8266 handles it without problems, ESP32 crash. Why?

[MBerka]

@sblantipodi I had it on 8266, so any differences you see may come down to other platform-specific issues like memory management.

[sblantipodi]

@MBerka this happen even with an empty mqtt callback function, so I don't think that it's a problem of memory management.

[sblantipodi]

ok it happen even with a simple sketch, fast messages for some minutes crashes the client. 30 message per second is enough to crash the client, this does not happen on esp8266, same code.

[MBerka]

@sblantipodi I meant management that could differ between 8266 and 32, at a lower level. Your code is the same and pubsubclient is the same on each of those, but other firmware could differ by target. The other less likely possibility is an obscure hardware flaw affecting just that 32. Either way, something might be preventing the 32 from clearing the message backlog as often? Saying any more than that would take a trace.

[sblantipodi]

@MBerka I can't say more. I have a simple sketch, the easiest possible. I subscribed to a topic that receives 30 message per second, ESP32 crash, ESP8266 not.

[sblantipodi]

adding a simple delay(5) in the main loop fixed the crash, ESP32 is so buggy when compared to ESP8266.

[MBerka]

@sblantipodi Alright, thanks for sharing that. I deal with an 8266 that has some unavoidable delays built in, but its problem has been keeping up with the flow rather than checking it too quickly. Good to know that there are possibly limits on both sides, for the ESP32.

[sblantipodi]

I correct myself, fast messages still crash the ESP32

[danidip]

Same here, simple sketch it hangs only when i publish too many messages on esp-01 and esp-12E, no problem with callback, i've read somewhere else it's related with espclient since pubsub supports either espclient or etherclient (this one without a problem apparently)