Telegram lost when writing multiple telegrams simulations

[heleon19]

Hi Supergiovane

I’m using Node-RED V1 with the new asynchronous flow processing, where the node messages are passed by “setImmediate”. When I try to send multiple KNX write telegrams simulations often one of them is not written to the KNX-Bus. When I add an additional delay of 50ms for each telegram, then the behaviour is better. Do you have an Idea to ensure all telegrams are written, even I pass them simulations to your KNX node?

Many thanks for your support! Heleon19

[Supergiovane]

Hello Heleon19, How do you assume, that the telegrams are not written? Do you monitor the KNX Bus using ETS? That's important to know. And please, can you post a screenshot of your flow and, if possible, the code of your flow? And, what appens if you stop using Async, bu putting runtimeSyncDelivery: true in the Node-RED config? Thanks.

[heleon19]

Yes I'm using ETS for monitoring the Bus. I had the problem yesterday with an RGB LED, one of the colors was not written. I do not have access to that system right now, therefor I did an other test with the system at my home.

Actualy here I'm using an Node-RED version below V1, therefor I prepared a function node for simulating the pehavior.

When I trigger the flow below, sometimes telegrams are missing, as you can see at the print screen of the GroupMonitor.

[ { "id": "c81a089b.504928", "type": "inject", "z": "ea717d77.07fe", "name": "", "topic": "", "payload": "true", "payloadType": "bool", "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "x": 95, "y": 105, "wires": [ [ "69ee379c.bce698" ] ] }, { "id": "69ee379c.bce698", "type": "function", "z": "ea717d77.07fe", "name": "setTimeout 0ms and send 5 messages", "func": "\nsetTimeout(() => {\n node.send(msg);\n}, 0);\nsetTimeout(() => {\n node.send(msg);\n}, 0);\nsetTimeout(() => {\n node.send(msg);\n}, 0);\nsetTimeout(() => {\n node.send(msg);\n}, 0);\nsetTimeout(() => {\n node.send(msg);\n}, 0);", "outputs": 1, "noerr": 0, "x": 330, "y": 105, "wires": [ [ "3e9a8b3f.9f2f24" ] ] }, { "id": "3e9a8b3f.9f2f24", "type": "knxUltimate", "z": "ea717d77.07fe", "server": "39ebf231.b0aebe", "topic": "1/0/0", "dpt": "1.001", "initialread": false, "notifyreadrequest": false, "notifyresponse": false, "notifywrite": true, "notifyreadrequestalsorespondtobus": false, "notifyreadrequestalsorespondtobusdefaultvalueifnotinitialized": "0", "listenallga": false, "name": "", "outputtype": "write", "outputRBE": false, "inputRBE": false, "x": 560, "y": 105, "wires": [ [] ] }, { "id": "39ebf231.b0aebe", "type": "knxUltimate-config", "z": "", "host": "192.168.2.250", "port": "3671", "physAddr": "1.1.250", "suppressACKRequest": false, "csv": "", "KNXEthInterface": "Auto", "KNXEthInterfaceManuallyInput": "", "statusDisplayLastUpdate": true, "statusDisplayDeviceNameWhenALL": true, "statusDisplayDataPoint": false } ]

Example flow, sends 5 messages in parallel:

Missing telegrams:

Would the connection option minimumDelay help? Is the knxjs library able to process a queue of write telegrams?

// wait at least 10 millisec between each datagram
  minimumDelay: 10,

[heleon19]

And the same test without setTimeout:

[ { "id": "2e9996ca.cac28a", "type": "function", "z": "ea717d77.07fe", "name": "send 5 messages", "func": "node.send([msg, msg, msg, msg, msg]);", "outputs": 5, "noerr": 0, "x": 270, "y": 180, "wires": [ [ "3e9a8b3f.9f2f24" ], [ "3e9a8b3f.9f2f24" ], [ "3e9a8b3f.9f2f24" ], [ "3e9a8b3f.9f2f24" ], [ "3e9a8b3f.9f2f24" ] ] } ]

[heleon19]

And here with return [msg, msg, msg, msg]

All variants show the same behavior.

[Supergiovane]

Hello Version below 1 should not cause any issue, as well as V1.02 (i'm using this version without issues. Your settimeout code are all set to zero, so the messages are sent all at the same time. As i can see, you're using an IP Interface. If the IP Interface is capable of multicast routing, please use the multicast KNX address 224.0.23.12 instead of the interface's IP 192.168.2.250. Multicast routing is way better. If your IP Interface is not capable of Multicast routing, please try to activate the "Suppress the ACK request", as per the image below:

Be aware, that KNX uses UDP in any cases. If your ethernet switch has some other works to do (other traffic), it can be possible that it loses some UDP packets. Be sure to have also disabled any multicast flood protection on the ethernet switches (if any), because you are really "flooding" the LAN with UDP packet.

The right procedure to "flood" a KNX node with telegram, is to add a delay in front of it:

Please import this code below, and put it in front of the knx node, as you can see in the image above. Then try again and let me know.

[{"id":"f8948492.880cd8","type":"delay","z":"eb97b356.202078","name":"","pauseType":"delay","timeout":"150","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":290,"y":360,"wires":[[]]}]

[heleon19]

Unfortunately, my interface can't multicast. I have activated the "Suppress the ACK request", no change here either.

The delay of 150ms for each message will certainly help to reduce the number of errors. For a single RGB logic this would certainly help a bit. So what is whom do several RGB logics want to send at the same time? Shouldn't the limitation already be implemented in the KNX Lib? From the knxjs API description I saw that there is a corresponding option, what if you could set this limitation in the config node?

The proposed delay didn't work, so I changed it to one below: [ { "id": "bd7e9553.41f628", "type": "delay", "z": "ea717d77.07fe", "name": "", "pauseType": "rate", "timeout": "150", "timeoutUnits": "milliseconds", "rate": "1", "nbRateUnits": "0.15", "rateUnits": "second", "randomFirst": "1", "randomLast": "5", "randomUnits": "seconds", "drop": false, "x": 625, "y": 105, "wires": [ [ "3e9a8b3f.9f2f24" ] ] } ]

[Supergiovane]

Yes, sorry, you're right. The code i pasted was wrong. Be aware that the KNX specs, suggests up to max 50 messages per seconds, that means that the min delay is 20millisecs, not 15milliseconds. V 1.1.16 has a delay of 60millisecs added and will be out, i think, in about 30 minutes. It handles all delay by his own, so you don't need the delay node anymore.

Please let me know if you'll have other issues. Cheers and thank you for using my node!

[heleon19]

Thank you, for this simple test it works. I'll test it on the other system tomorrow, and report to you.

[waldbaer]

I am facing the same issue. Therefore I created this test flow using the new knx-ultimate version 1.1.16.

[{"id":"7d459a68.9a213c","type":"function","z":"8e164a6.71881b8","name":"gen outputs","func":"let outputs = []\n\nfor(let i = 0; i<0x80; i++) {\n outputs.push({payload: i});\n}\nreturn [ outputs ];","outputs":1,"noerr":0,"x":370,"y":200,"wires":[["13f35500.61af9b","df0cc0cf.ebe25"]]},{"id":"5a20ca24.cfe83c","type":"inject","z":"8e164a6.71881b8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":200,"y":200,"wires":[["7d459a68.9a213c"]]},{"id":"df0cc0cf.ebe25","type":"knxUltimate","z":"8e164a6.71881b8","server":"e307417e.b4e0c","topic":"2/2/2","dpt":"5.004","initialread":false,"notifyreadrequest":false,"notifyresponse":false,"notifywrite":false,"notifyreadrequestalsorespondtobus":false,"notifyreadrequestalsorespondtobusdefaultvalueifnotinitialized":"0","listenallga":false,"name":"","outputtype":"write","outputRBE":false,"inputRBE":false,"x":550,"y":200,"wires":[[]]},{"id":"e307417e.b4e0c","type":"knxUltimate-config","z":"","host":"192.168.0.40","port":"3671","physAddr":"15.15.22","suppressACKRequest":false,"csv":"","KNXEthInterface":"Auto","KNXEthInterfaceManuallyInput":"","statusDisplayLastUpdate":true,"statusDisplayDeviceNameWhenALL":true,"statusDisplayDataPoint":false}]

When I trigger the flow the first time I see all messages in the KNX trace with the payloads 0x00..0x80. But with every next trigger of the flow no more message are transmitted on the KNX bus. Only a re-deploy currently helps. And sometimes I get the following error: "knxUltimate: Node df0cc0cf.ebe25 has been disabled due to loop detection. Check your flow's design or use RBE option."

Environment: nodered 1.0.3, KNX interface: knxd with TUL stick on rpi.

[Supergiovane]

Why are you trying to flood your poor knx bus? The loop protection prevents exactly this. Please take a look at the readme in the github. Then please use a real knx/eth ip router, not a knxd.

[waldbaer]

The provided flow is just a test scenario.

The real use case was implementation of lighting scenes like 'all lights off' on node-red. In this case I need around 50 messages on the bus. I know that this could also be done by programming the knx devices and use a single knx message. But for prototyping and flexibility reasons it should be possible to do. 

What's the big difference between knxd and 'real' hardware routers? 

[Supergiovane]

Hello with knxd have had some issue in the long run with hangs and stability issues. The connection between Ethernet and KNX BUS is the most important things and should be done by a dedicated IP Router. Please split your sequence of 50 messages in two or three steps, separated with at least 1 second each step, or put, just behind your function, a delay node and set, for example, max 1 telegram each 150 msecs. This avoids flooding the bus. If you don't do so, the auto flood protection of the knx-ultimate kicks in, to avoid flooding your KNX Bus, loosing KNX datagrams and to make datagrams of your real KNX devices not timing out.

Let me know.

[waldbaer]

I agree that nobody should flood the 9.6kBaud KNX bus with hundreds of messages. But for me the error message "loop prevention" is a little bit misleading. Better would be a error message like "overload situation detected".

[Supergiovane]

.. but... wait a moment. You’re incurring in a loop protection, not in a flood protection. If you need to switch all lights off, you need to use knx-ultimate as a universal node and to pass the group addres you want to switch off, along with the payload. In your sample, you set only one node with 2/2/2. If a loop protection kicks in, it means that you have two node with the same group address, link together, even if there are other nodes inbetween. When it kicks in, if you go through you flows, you’ll see exactly where it happens, because the affected node will have a status indicating that the node has been disabled. Please verify that again.