eclipse / mosquitto

Eclipse Mosquitto - An open source MQTT broker
https://mosquitto.org
Other
8.92k stars 2.37k forks source link

Details of persistence implementation #1855

Open Stasik0 opened 3 years ago

Stasik0 commented 3 years ago

Dear community,

I'd like to ask about the implementation of persistence mechanisms in mosquitto: in our use case we really care about persistence of QoS1 messages, i.e. there is a hard requirement not to loose any of the messages once once a PubAck has been sent.

Currently we test with the following config: autosave_interval 1 autosave_on_changes true persistence true

The broker behaves as expected during unpredicted shutdowns etc, however I'd like to ask for corner-cases. Can you tell me, whether the PubAck is really sent out once the messages has been saved on the drive? When exactly, e.g. after flushing the file buffers? Maybe you can also tell me the location in the code?

ralight commented 3 years ago

The persistence implementation isn't particularly optimised for this very frequent saving of data. Persistence is something I'm planning on looking at in the version after next.

The way it operates at the moment is that for each iteration of the network loop the autosave changes count is checked, and if it exceeds the autosave_interval option, then messages are persisted to disk. This means that when a publish is received there isn't a strict relationship it being persisted to disk and the puback being sent. There is however a pretty limited amount of time between those two events.