search

smartrent / tortoise311

An MQTT 3.1.1 Client written in Elixir
Apache License 2.0
11 stars 8 forks source link

Reconnection reaching max file descriptor limit #52

Open FlorianGerhardt opened 1 year ago

FlorianGerhardt commented 1 year ago

System Info Elixir 1.14.3 and 1.15.4 OTP 25.2.2 and 26.0.2 Nerves 1.10.3 with nerves_system_rpi4 1.23.2

It seems like tortoise311 is under certain circumstances opening files and not closing them again, resulting in too many open files and system failure. We currently have issues with our mqtt server and Tortoise is constantly trying to reconnect to it, which seems fine for most of our devices, but we have two devices running with a mobile router and only these 2 are so far affected by that issue, so this might be related to the connection.

By checking /proc/pid/fd of my application there are around 170 open files after about 1hr uptime of the device and they are constantly increasing, it seemed to me it was increasing at the same pace as I got the reconnect message in the console from Tortoise. After stopping Tortoise it dropped down to around 60 and not increasing anymore.

iex(30)> cmd("ls -l /proc/83/fd")
lrwx------    1 root     root            64 Oct  5 10:30 109 -> socket:[7579]
lrwx------    1 root     root            64 Oct  5 10:33 108 -> socket:[5369]
lrwx------    1 root     root            64 Oct  5 10:33 107 -> socket:[7589]
lrwx------    1 root     root            64 Oct  5 10:30 106 -> socket:[5357]
lrwx------    1 root     root            64 Oct  5 10:30 105 -> socket:[7600]
lrwx------    1 root     root            64 Oct  5 10:30 104 -> socket:[7556]
lrwx------    1 root     root            64 Oct  5 10:27 103 -> socket:[7566]
lrwx------    1 root     root            64 Oct  5 10:30 102 -> socket:[7546]
lrwx------    1 root     root            64 Oct  5 10:30 101 -> socket:[7539]
lrwx------    1 root     root            64 Oct  5 10:27 100 -> socket:[6436]