Unable to receive payload greater than 65536 bytes

[shadowfighterCA]

I am trying to receive around 2 MB of payload via MQTT on esp32. But maximum packet size that I can set is 65536 via client.setBufferSize() function. The argument of this function is a uint16_t hence i am unable to set receiving packet size more than 65536. Such large payloads can be easily received by ESP-IDF. Can you help me with this ? I tried setting it up higher than 65536 but it simply does not work.

Code for reference -

#include <WiFi.h>
#include <PubSubClient.h>

// WiFi
const char *ssid = "hello"; // Enter your WiFi name
const char *password = "world";  // Enter WiFi password

// MQTT Broker
const char *mqtt_broker = "broker.emqx.io";// broker address
const char *topic = "esp32/test"; // define topic 
const char *mqtt_username = "emqx"; // username for authentication
const char *mqtt_password = "public";// password for authentication
const int mqtt_port = 1883;// port of MQTT over TCP

WiFiClient espClient;
PubSubClient client(espClient);

void setup() {
 // Set software serial baud to 115200;
 Serial.begin(115200);
 // connecting to a WiFi network
 WiFi.begin(ssid, password);
 while (WiFi.status() != WL_CONNECTED) {
     delay(500);
     Serial.println("Connecting to WiFi..");
 }
 Serial.println("Connected to the WiFi network");
 //connecting to a mqtt broker
 client.setServer(mqtt_broker, mqtt_port);
 client.setCallback(callback);
client.setBufferSize(65536);
 while (!client.connected()) {
     String client_id = "esp32-client-";
     client_id += String(WiFi.macAddress());
     Serial.printf("The client %s connects to the public mqtt broker\n", client_id.c_str());
     if (client.connect(client_id.c_str(), mqtt_username, mqtt_password)) {
         Serial.println("Public emqx mqtt broker connected");
     } else {
         Serial.print("failed with state ");
         Serial.print(client.state());
         delay(2000);
     }
 }
 // publish and subscribe
 client.publish(topic, "Hi EMQX I'm ESP32 ^^");
 client.subscribe(topic);
}

void callback(char *topic, byte *payload, unsigned int length) {
 Serial.print("Message arrived in topic: ");
 Serial.println(topic);
 Serial.print("Message:");
 for (int i = 0; i < length; i++) {
     Serial.print((char) payload[i]);
 }
 Serial.println();
 Serial.println("-----------------------");
}

void loop() {
 client.loop();
}

[superbool]

You can split the packets to small ones then send one by one . In you esp32 when received try to merge the packets.

https://mqtt.org "It is designed as an extremely lightweight publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth", it's not suit to transport large data.

[shadowfighterCA]

hey, thanks for response. As per mqtt specs maximum payload can be 256 MB. And I have received more than 10 MB payload on the same hardware (ESP32) using native framework (ESP IDF). So as far as hardware and protocol goes, it is possible to receive more than 10 MB payload. I was asking if someone has modified this library for ESP32. Arduino framework uses esp idf in backend so doing this is possible.

[HamzaHajeir]

Hi

Please feel free to checkout my maintained fork of H4AsyncMQTT. Just keep in mind the availability of RAM to handle continuous TCP fragments.