Messages dropped. potential Mosquitto.db memory leak.

[bhavinjethra]

Hi guys, I have a situation of memory leak while using mosquitto 2.0.14 on Ubuntu 18.04. Setup: Paho Mqtt publisher 1.5.0 that publishes 20 messages per second to the local mosquitto broker with the QoS level 2. The local broker then forwards the data to an external broker. Persistent is set to true and cleansession is set to false.

Scenario: There is 1 hour forced network outage between the local mosquitto broker and the external mosquitto broker. Upon reconnection, the connection is established seamlessly.

Outcome: There is intermittent loss of data from the mosquitto.db for the entire duration.

Debugging logs trimmed analysis: The number of incoming publish messages to the local broker: ~72000 (conforms to the 20 messages per sec) The number of outgoing publish messages to the external broker: ~48,000

$SYS/broker/publish/messages/dropped value seems to be increasing consistently after a certain point.

Local broker's mosquitto.conf:

tls_version tlsv1.2

persistence true persistence_location /var/lib/mosquitto/ autosave_interval 1 log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

connection external-mosquitto-bridge cleansession false max_queued_messages 0 max_inflight_messages 10000 log_type all log_timestamp_format %Y-%m-%dT%H:%M:%S address 198.168.1.100:6060 topic MQTT_Test both 2

External mosquitto broker's .conf:

tls_version tlsv1.2

persistence true persistence_location /var/lib/mosquitto/ autosave_interval 60 listener 6060 0.0.0.0 log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d allow_anonymous true

max_queued_messages 0 max_inflight_messages 10000 log_type all log_timestamp_format %Y-%m-%dT%H:%M:%S

How can I help you with more information? If not, how can I tweak my settings to make this thing fly?

Thank you in advance.

[karlp]

do you really mean autosave 1? that's incredibly frequentlty to be resaving the entire database....

[bhavinjethra]

I updated the autosave interval to 60. Also, I'm currently experimenting how long does it take before the mosquitto broker starts losing data. I've been letting my system run for the past 26 hours, and I'm subscribed to '$SYS/#' using mosquitto_sub. I don't see any dropped message this time so far. The mosquitto.db has reached nearly 1.7 GB at the moment.

I'm planning to reconnect the external broker in a few hours, and hope to see all messages going through.

[bhavinjethra]

update: Upon reconnecting the external broker, I see lots and lots of messages not crossing through. Any ideas? I'm aiming for 0% data loss.

[bhavinjethra]

CLosing this as I realized there were other pieces involved related to MQTT that were causing the issue.