mqtt subscription rate

[EurekaAssociation]

Hi Stefan,

Thanks again for the static ip fix. I've been building on your library for a few days now and I've noticed that if many messages are sent in rapid succession, not all are treated by the library. (I suspect that the payload is overwritten by the value of the next message before the process has time to finish processing the original payload)

I have placed a rate limiter in my node-red flow to slow down the rate of messages posted to the mqtt broker, but it would be great if this was directly handled by the subscribing client. (1 msg per 0.05 seconds is the limit for my PLC to handle, but for 32 outputs to switch individually this takes 1.6 seconds...)

It is probably nit-picking, but I like to stress-test my implementation to know it can handle it when push comes to shove. :)

Thanks, Yorick

[stefandreyer]

Hi Yorick, I'm happy for your stress testing. There can be many reasons for this behavour. 1st Question: what QoS Level do these Messages have? 2nd Does all Topics fit into the borders of COM_CONFIG? 3rd can you make an Wireshark sniff fore These conditions? 4th is it possible that you change states of Topics rapidly? mayby if you Change on, off, on just the off is lost?

There are some buffers in the library. If they are full and the packet is QoS = 0 then the paket is lost. In my stress test FB in the library sends and resives 120 pakets/second. And it is possible to rescive more then on packet perPLC cycle, but the PLC application can not handle more then one packet per cycle. This is because if you recive one publish packet(Topic: 'switch') the client doesn't know if it comes because the the subscription to'#' or to 'switch/#'. This have to handle the subscription FBs. So every Topic has to be checkt by every subscription FB if the Topic fits to the subscription string. And this comes near to your suspection, one by one packet is handeld per cycle --> cycles per second --> max packets per second.

So set QoS to at least 1 and no Data should be lost.

BR Stefan

[EurekaAssociation]

Hi Stefan,

The QOS is 2, it is all in the same topic "PLC/OUTPUTS". I'll check out the states and have a go at wireshark as well.

I'll try my best to have more info in the coming days, but starting tomorrow I'll be painting walls and installing laminate floor all day long in our new house, so there won't be much time left for 'spielerei' ;)

[stefandreyer]

Hi Yorick,

it is all the same topic, ok, how do the payload look like? if you use something like OUTPUT1:ON and OUTPUT1:OFF maybe it is a race condition on broker side, so that the order of the packets is beeing changed. And if you paintig walls, don't forget the copper in the walls...

BR Stefan

[EurekaAssociation]

Hi Stefan,

The payloads are DO0_1=1, DO1_3=1 , etc..., no repeat of on and off messages.

[stefandreyer]

OK,

now a network sniff would be interesting.

BR Stefan

[stefandreyer]

Hi Yorik,

do you use the retain flag?

BR Stefan

[EurekaAssociation]

I've turned off the retain flag for publish and subscribe.

[stefandreyer]

Hi Yorick,

I think the broker can't store different payloads to one topic(it doesn't have a stack for each client/topic). So if you send rapidly many packets to one topic, only the last packet is published to the subsribers. So if the broker gets a new payload for the topic while the last one wasn't publish, it publish the new one. This we would see on a sniff

BR Stefan

[YorickErnst]

I'll try to do that as soon as possible. (I just realised I used my company account to create this issue :) )

[stefandreyer]

Hi Yorick

please try something like Topic:PLC/OUTPUTS/DO0_1 Payload:1 or Payload:0

BR Stefan

[stefandreyer]

Hi Yorick,

check if PublishReciveBufferCountFull in the HANDLE_MQQT FB is increasing, if so, the Buffer were full, but on QoS > 0 the broker hvae to repeat…

BR Stefan

[YorickErnst]

Hi Stefan,

I managed to find half an hour to test further.

Rename this pdf to .cap for wireshark capture: attempt.pdf

I sent 4 times 3 payloads at 0.01msg/s. Sometimes 2 payloads were read, other times only 1. (Each time the 3 payloads were succesfully sent to mqtt broker)

The PublishReciveBufferCountFull is 0.

Kind regards, Yorick

[stefandreyer]

Hi Yorick,

thanks for the sniff. i see the QoS is 0, you said it would be 2? I have made a test FB for testing such issues, but now i have Problems, that mosquitto Queue to Client ist to small…

You'll hear from me.

BR Stefan

[YorickErnst]

Hi Stefan,

You are correct. I was using QOS 2 in plc but not in external publisher. I changed it to QOS 2, but the issue remains.

[stefandreyer]

Hi Yorick,

please test new Version

BR Stefan

and yes i am thinking About Performance optimation … ;)

[YorickErnst]

Hi Stefan,

I've just finished the move to our new house (still some boxews to unpack though), and I've tested your improvements just now.

Great work, it is now stable at 0.02 messages / second for at least 4000 messages. For home automation this seems to be more than adequate. :)

For smaller amounts of messages I even saw a stable situation at 0.016 msg/s, but let's not tempt fate for 0.004 msg/s :D

Thanks for all your work!