bytebeamio / rumqtt

The MQTT ecosystem in rust
Apache License 2.0
1.65k stars 255 forks source link

rumqttc: Error when receiving messages larger than 10240 bytes. #924

Closed Meisterian closed 4 days ago

Meisterian commented 5 days ago

Expected Behavior

The maximum payload size is 65kb. Receiving messages larger than 10240 bytes shall not cause an error. Trying to set the receive maximum payload (two times) like this is useless:

mqttoptions
    .set_transport(transport)
    .set_keep_alive(Duration::from_secs(30))
    .set_receive_maximum(Some(u16::MAX))
    .set_connect_properties(ConnectProperties {
        receive_maximum: Some(u16::MAX),
        ..Default::default()
    });

We process the same messages using mosquitto, paho-c, paho-cpp and paho-golang. All those libraries works without any problems. The only mqtt limit that I can find on internet; is that the topic can't be bigger that 10240 bytes.

Current Behavior

Messages larger than 10240 bytes cause an error and the client will be disconnected.

Failure Information (for bugs)

I tried to find this magic number in your code; but I haven't found it yet.

Context

Using version 0.24.0

Failure Logs

Regards, Christian

swanandx commented 5 days ago

hey @Meisterian , receive_max in MQTT v5 is more like max inflight queue ( i.e. max messages connect can be accepted at once ), you should set max_packet_size !

Meisterian commented 4 days ago

Thanks @swanandx, that solved my problem! I will close this issue. But one question; why do you set this value so low by default? It is a great feature to be able to restrict the maximum payload size, but shouldn't this be set to None as default?

swanandx commented 3 days ago

why do you set this value so low by default?

i don't have context of it, maybe while building it 10kb would have been found as reasonable default for IoT use cases.

We restrict max payload size even for mqtt v3 client so that we have boundedness over resource consumption. Hence, in v5 as well we set it to some value ( 10kb ) instead of setting it to None.

danbachar commented 2 days ago

sorry I have a similar issue, but there's something I don't get here - isn't the max payload size for MQTT around 256 MB?

swanandx commented 2 days ago

sorry I have a similar issue, but there's something I don't get here - isn't the max payload size for MQTT around 256 MB?

yes it is, but we decided to add additional boundedness in the client, you can always increase the limits when building mqttoptions.

danbachar commented 1 day ago

the thing is, every time I increase the client max buffer size above 65536 I get a disconnect on the write function, and an error in the broker logs, also when using your broker (rumqttd). Is there something I am overlooking here?

swanandx commented 1 day ago

the thing is, every time I increase the client max buffer size above 65536 I get a disconnect on the write function, and an error in the broker logs, also when using your broker (rumqttd). Is there something I am overlooking here?

Assuming by max buffer size you mean max payload size, please check what is the max payload size configured in rumqttd.toml file as well! It might be the reason for disconnection ( payload exceeds limit error will be logged )