eclipse / mosquitto

Eclipse Mosquitto - An open source MQTT broker
https://mosquitto.org
Other
8.88k stars 2.36k forks source link

Rate limit sending queued up messages from bridge? #1359

Open beville opened 5 years ago

beville commented 5 years ago

We've recently seen a scenario where the mosquitto bridge had queued up 20000 (a limit we set up the config) outgoing QoS1 messages. This is on a mobile device with limited/spotty connections. When the bridge was finally able to connect to to the remote broker, it overwhelmed it with a huge tidal wave of messages from the queue.

Does there exist a way to rate limit the outflow of queued up messages?

ckrey commented 5 years ago

Try max_inflight_messages set to e.g. relatively small number (10 or 20)

https://mosquitto.org/man/mosquitto-conf-5.html

beville commented 5 years ago

@ckrey Because we have multiple local mqtt clients sending messages (with sensor data that we don't want to lose) to the single mosquitto bridge, we want the bridge to hold onto all the messages until the cell connection is restored.

Unfortunately, max_inflight_messages is just a hard limit on the count of messages, and doesn't seem to affect the rate at which the messages are sent once the connection is made.

We were hoping to delegate the awareness of remote broker connectivity and message queuing to mosquitto. I suppose if the rate limiting doesn't exist yet, we will have to look into either patching mosquitto, or rolling our own bridge.

@ralight Off hand, how complex would it be to add this feature?

cimplart commented 1 year ago

I join in with this issue. We need to limit the rate at which mosquitto bridge dumps locally retained topics to an external broker.