search

eclipse / mosquitto

Eclipse Mosquitto - An open source MQTT broker
https://mosquitto.org
Other
8.93k stars 2.37k forks source link

Connection and reception are slowed by FIN_WAIT #2101

Open bahientn9x opened 3 years ago

bahientn9x commented 3 years ago

My Mosquitto sometimes connects very hard and sends it too. After checking everything, mosquitto is still running normally, cpu and memory don't spike. However, when checking with netstat I see FIN_WAIT1 queue with up to 2000 connections. And that caused the mqtt connection to be affected and I specifically checked there was no sign of being attacked from a specific ip, each connection in it was from a different ip, I think that is when my client disconnects mqtt. Because FIN_WAIT1 appears when the client sends a packet to the server to disconnect. Is it because of the buffer overflow of tcp port ? My client volume is always around 8000. My configuration is 4 core / 4gb ram.

ralight commented 3 years ago

FIN_WAIT1 means "the TCP port is closing", in other words the client or server is closing the connection but it has yet to complete.

Could I ask what version of Mosquitto you are running, and on what platform? What have you configured as the maximum number of sockets allowed for Mosquitto? I mean the ulimit type limit in the system, not a Mosquitto configuration parameter.

bahientn9x commented 3 years ago

FIN_WAIT1 means "the TCP port is closing", in other words the client or server is closing the connection but it has yet to complete.

Could I ask what version of Mosquitto you are running, and on what platform? What have you configured as the maximum number of sockets allowed for Mosquitto? I mean the ulimit type limit in the system, not a Mosquitto configuration parameter.

I am running on ubuntu 20 lts system and latest mosquitto 2.0. All of them just got reinstalled a few days ago. Here is my configuration Screen Shot 2021-02-25 at 17 06 37

mosquitto.conf Screen Shot 2021-02-25 at 17 07 45

ralight commented 3 years ago

Thank you, it looks like you're nowhere near the limit so it must be something else.

My Mosquitto sometimes connects very hard and sends it too.

I'm sorry I don't quite understand what you mean - is it that the clients suddenly get disconnected?

Do you have any logs from mosquitto? Is /etc/mosquitto/passwd accessible by the mosquitto user? Does anything similar happen if you send a SIGHUP signal to the mosquitto broker?

bahientn9x commented 3 years ago

Thanks for the reply and sorry for my bad english language.

I'm sorry I don't quite understand what you mean - is it that the clients suddenly get disconnected?

I mean, when the FIN_WAIT queue is high, my clients take a long time to connect to the server, and sending the messages takes a lot of time. Like when the network connection is not stable. Only when the FIN_WAIT1 queue is very low will connections be established quickly. I am not getting any logs from mosquitto.log and my clients is still accessing it normally before