search

emqx / MQTTX

A Powerful and All-in-One MQTT 5.0 client toolbox for Desktop, CLI and WebSocket.
https://mqttx.app
Apache License 2.0
3.8k stars 438 forks source link

[Feature] Add the option to automatically collapse large messages #1232

Closed TommyDuc closed 3 months ago

TommyDuc commented 1 year ago

Motivation

Large messages (read about ~100MB) are fetched and displayed as is by MQTTX. Not only is this not practical but it also makes the application lag and even crash.

Detailed design

There should be a limit for the maximum length for messages being displayed. If that limit was exceeded then the message should be automatically collapsed.

Red-Asuka commented 1 year ago

Thank you for bringing this to our attention. We hadn't considered the display issue with large messages before. I will take your suggestion into account and work to implement it as soon as possible.

TommyDuc commented 1 year ago

Thanks @Red-Asuka !

ysfscream commented 1 year ago

@TommyDuc Hi, before we optimize this, could I ask you two questions?

  1. What is the scenario that requires such a large size of data?
  2. What is the current performance and show of MQTT X when receiving and sending this large amount of data?

We want to be able to reproduce this when optimizing fully. Thanks

TommyDuc commented 1 year ago

@ysfscream

  1. The scenario is that the MQTT broker is used to transfer files (read firmware updates). We know that MQTT was not meant to transfer files but it does support messages up to 256MB :) and so it was quick and cheap way to do it.
  2. For uploading I haven't tried it but when file-messages, which are often retained, then opening MQTT X on the file topic causes it to freeze for multiple minutes or even crash.
ysfscream commented 1 year ago

OK. Thanks for your details. We will optimize this data display, but you can wait for the large file transfer function of EMQX, which will work very well for you!

ysfscream commented 3 months ago

It's updated to https://github.com/emqx/MQTTX/releases/tag/v1.10.0, welcome to try and thanks again.