Enhancement Request: Option to set the order of offline messages and more

[asutoshg]

Please fill out the form below before submitting, thank you!

• [x] Bug exists Release Version 1.1.0 ( Master Branch)
• [x] Bug exists in Snapshot Version 1.1.1-SNAPSHOT (Develop Branch)

If this is a bug regarding the Android Service, please raise the bug here instead: https://github.com/eclipse/paho.mqtt.android/issues/new

Presently delivery of offline messages takes place in the order it was produced and only single buffer of max size 5000 msg is supported. In situations where data rate is high and network is down for prolonged duration, there is a possibility of loss of data due to small size and number of offline buffer. Also, after connectivity is restored it may take quite a long time to clear the backlog. In application where latest data is of higher relevance, the application will have to wait many more minutes, even after connectivity is restored, before it starts seeing useful data.

I would like to have following enhancements to support my use case:

• Possibility to set the order of offline message to be sent in a buffer(FIFO/LIFO)
• Possibility to have N number of active offline buffers with size 5000. Max number is set by publisher but buffer is created if required.
• Possibility for subscriber to archive/delete buffers if clearing them does not make sense (based on time elapsed, rules, brute force)
• Possibility to set order of offline buffers to be chosen for clearing (FIFO/LIFO)

Thanks

[jpwsutton]

Thanks for the enhancement request @asutoshg

• LIFO sounds doable, I think the best way will be to add a setLastInFirstOut to the DisconnectedBufferOptions, where the default is false. So the normal behaviour would be FIFO, but with this enabled it would be LIFO.
• There isn't actually a maximum buffer size, the default is 5000 messages, but you can use DisconnectedBufferOptions.setBufferSize to change it to any number you like (above 0 though). The only restriction currently is probably either the amount of RAM in your computer, or the Java Max Int limit (2,147,483,647) - 8 (http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/00cd9dc3c2b5/src/share/classes/java/util/ArrayList.java#l191)
• At the moment, we offer a limited control over the buffer through MqttAsyncClient.getBufferedMessage, and MqttAsyncClient.deleteBufferedMessage. I guess we could add something that returns the entire buffer though?
• What would the use case be for having multiple buffers over one very large buffer?

[asutoshg]

Thanks for our response @jpwsutton Many smaller sized buffers would give me flexibility to choose which one to publish ,expunge ,archive Also, time to clear small buffer would be less compared time taken to clear one large buffer (be it FIFO or LIFO). In order to accommodate more messages multiple such buffers will be required, just in case the network is down for prolonged duration.

[shaikhrahil]

@jpwsutton would there be a python wrapper for the max offline buffer size?

[olad32]

I agree, the ability to have multiple buffers would allow to prioritize different types of messages.