search

peterprib / node-red-contrib-kafka-manager

Implement
GNU General Public License v3.0
22 stars 12 forks source link

network connection increase periodly #71

Open hnuzhoulin opened 8 months ago

hnuzhoulin commented 8 months ago

First, thanks this nodered kafka node.I can transfer data from mqtt to kafka or kafka to mqtt easily. I face a problem when using nodered and this kafka node. the nodered dashboard 1880 periodic can not access,if I restart nodered process,it works again.then I debug it.

I noticed that the number of TCP connections had risen from 400 to over 4,000 in the last four hours. All these new TCP connections were to Kafka. Here, Node-RED acts as a Kafka producer, so I couldn't find any useful information on the Kafka side. Looking back at the historical monitoring, I noticed that each time there was a nodered failure, there was this phenomenon of connection counts skyrocketing to over 4,000.

I verified if the data flow in the processes behind Node-RED was interrupted when face this problem. Yes,data forward works well. although the CPU and load on the server hosting Node-RED were high and increase , maybe the load let nodered dashboard not work.

I discuss this in nodered forum,they suggest here for help.

What can I do to find the problem. BTW,it works well six month until last week.yes,we add more device to mqtt.Thanks.

the attached file is my nodered flow which exported. flows202312131652.json

peterprib commented 8 months ago

I think I have noticed and fixed the issue with my big rebuild. Basically a bug in check if path available and failing to cleanup test connection properly. It is about finding time to test all. Underlying logic has had a massive restructure, hopefully with greater resilience to outlier states.