Memory problems on ESP8266

[lhtrevisan]

Hello my fellow friends!

I'm experiencing crashes when the ESP tries to connect to the broker using pubsubclient. After some tinkering I realized it could be a problem of lack of memory. I then proceeded to test a code containing only the wifi connection and MQTT connection, without the rest of the libraries and functions, while monitoring the free HEAP. That is the simplified code I used on the test:

Code

``` //---------------------------------------------------------- Bibliotecas #include #include #include #include //---------------------------------------------------------- NÚMERO DE SÉRIE E SENHA String NSERIE = "xxx00000"; //n° de série #define sendtopic "sendtopic" #define basetopic "basetopic" String fulltopic = "fulltopic"; unsigned long tAnterior_HEAP = 0; //---------------------------------------------------------- VARS : MQTT const char MQTT_IOT_ENDPOINT[] = "myendpoint"; const char* mqtt_username = "username"; const char* mqtt_password = "password"; const int mqtt_port = 8883; WiFiClientSecure espClient; PubSubClient client(espClient); unsigned long lastLog = 0; #define MQTT_MAX_PACKET_SIZE 512 #define MSG_BUFFER_SIZE (512) char msg[MSG_BUFFER_SIZE]; //---------------------------------------------------------- VARS: PORTAL LOCAL String ssid = "00000000000000000000000000000000"; String pass = "00000000000000000000000000000000"; bool connected_to_WiFi = false; bool connected_to_MQTT = false; //---------------------------------------------------------- VOID SETUP void setup() { Serial.begin(115200); Serial.println(""); Serial.println("----------------------------------------------------------"); Serial.println("Iniciando firmware."); Serial.print("HEAP: "); Serial.println(ESP.getFreeHeap()); ConfigEEPROM(); if (!WiFiconnect()){ } Serial.print("Setup finalizado. HEAP: "); Serial.println(ESP.getFreeHeap()); } //---------------------------------------------------------- LOOP void loop() { if (connected_to_MQTT == true && !client.connected()){ connectMQTT(); } else { client.loop(); } } //---------------------------------------------------------- FUNÇÃO: ConfigEEPROM //Busca dados guardados na EEPROM, como Rede, Senha, TokenBF, etc... void ConfigEEPROM(){ EEPROM.begin(512); //Initialasing EEPROM Serial.println("----------------------------------------------------------"); Serial.println("Buscando informações na memória..."); NSERIE = ""; for (int i = 0; i < 8; ++i){ NSERIE += String(char(EEPROM.read(450+i))); } Serial.print("NSERIE: "); Serial.println(NSERIE); ssid = ""; for (int i = 16; i <= 47; ++i){ ssid += String(char(EEPROM.read(i))); } Serial.print("SSID: "); Serial.println(ssid); pass = ""; for (int i = 48; i <= 79; ++i){ pass += String(char(EEPROM.read(i))); } Serial.print("Password: "); Serial.println(pass); Serial.println("----------------------------------------------------------"); EEPROM.end(); } //---------------------------------------------------------- FUNÇÃO: WiFiconnect //Tenta uma conexão com a rede com os dados obtidos da EEPROM bool WiFiconnect(){ Serial.println("Iniciando conexão Wi-Fi"); WiFi.mode(WIFI_STA); WiFi.begin(ssid, pass); int c = 0; while(c < 30){ if (WiFi.status() == WL_CONNECTED){ connected_to_WiFi = true; Serial.println(""); Serial.print("Wi-Fi Conectado. IP: "); Serial.println(WiFi.localIP()); espClient.setInsecure(); client.setServer(MQTT_IOT_ENDPOINT, mqtt_port); client.setCallback(mqttHandler); client.setBufferSize(MQTT_MAX_PACKET_SIZE); return connectMQTT(); } Serial.print("*"); delay(500); c++; } Serial.println(""); Serial.println("A conexão Wi-Fi falhou."); connected_to_WiFi = false; return false; } //---------------------------------------------------------- FUNÇÃO: connectMQTT bool connectMQTT(){ int c = 0; Serial.print("HEAP_Check: "); Serial.println(ESP.getFreeHeap()); while (!client.connected() && c < 10) { Serial.print("Conectando com o broker MQTT... "); if (client.connect(NSERIE.c_str(), mqtt_username, mqtt_password)) { Serial.println("Connectado"); connected_to_MQTT = true; fulltopic = basetopic + NSERIE; client.subscribe(fulltopic.c_str()); //TODO -- Dar ping pedindo atualização da informação return connected_to_MQTT; } else { Serial.print("Falha, rc="); Serial.print(client.state()); Serial.println(" tentando novamente em 2 segundos"); // Wait 2 seconds before retrying Serial.print("HEAP_Check: "); Serial.println(ESP.getFreeHeap()); delay(2000); } c++; } connected_to_MQTT = false; Serial.println("Tempo de conexão excedido."); //TODO -- Código de Erro return connected_to_MQTT; } void mqttHandler(char* topic, byte* payload, unsigned int length){ } ```

That is what is printed to the serial:

Serial

``` ******** Wi-Fi Conectado. IP: 192.168.1.142 HEAP_Check: 44040 Conectando com o broker MQTT... Connectado Setup finalizado. HEAP: 22840 ```

As you can see, the connection is using about 21kb of HEAP... My full code has, right before the connection, about 25k of free HEAP that is, obviously, fragmented, thus not being enough to stablish a successful connection.

There are 2 ways of solving my problem: 1 - Freeing or defragmenting some HEAP, so that it is enough to connect, but could fail after some time, as it would be running with low memory from the beginning. 2 - Reducing the HEAP memory consumed while connecting to the broker.

So I ask you about this memory consumption. Is it normal for this library to consume that much of the HEAP memory, or maybe I'm experiencing some weird interaction between my ESP board libraries and pubsub?

I'm coding on Arduino IDE. PubsubClient library is up to date(downloaded it again yesterday, to make sure it was not the problem).

[Garc964]

Hello @lhtrevisan, I did a heapcheck on my MQTT program:

Bibliotheken

include

include

include

include

include "Adafruit_LiquidCrystal.h"

include

include

Serial output 16:10:02.172 -> Connected to wifi. My address:192.168.1.20 16:10:02.205 -> Wednesday, January 19 2022 16:10:02 16:10:02.205 -> Connectie met de MQTT broker: 192.168.1.25 16:10:02.205 -> Verbonden met de MQTT broker ! 16:10:02.205 -> 16:10:02.238 -> sensor is a BME280 16:10:04.659 -> Wednesday, January 19 2022 16:10:04 16:10:04.758 -> Druk :1022.37 16:10:04.791 -> Druk64 :1022.38 16:10:04.791 -> Temperatuur:20.52 16:10:04.791 -> Vochtigheid:43.66 16:10:04.791 -> HEAP_Check: 250640

16:14:10.660 -> Wednesday, January 19 2022 16:14:10 16:14:10.759 -> Druk :1022.45 16:14:10.759 -> Druk64 :1022.41 16:14:10.792 -> Temperatuur:20.53 16:14:10.792 -> Vochtigheid:43.44 16:14:10.792 -> HEAP_Check: 250456

Arduino compile log Meerdere bibliotheken gevonden voor "WiFi.h" Gebruikt: /home/marc/.arduino15/packages/esp32/hardware/esp32/1.0.6/libraries/WiFi Niet gebruikt: /home/marc/Documenten/bin/arduino-1.8.18-linux64/arduino-1.8.18/libraries/WiFi Niet gebruikt: /home/marc/Arduino/libraries/WiFiEspAT Niet gebruikt: /home/marc/Arduino/libraries/WiFiNINA Bibliotheek Wire op versie 1.0.1 in map: /home/marc/.arduino15/packages/esp32/hardware/esp32/1.0.6/libraries/Wire wordt gebruikt Bibliotheek WiFi op versie 1.0 in map: /home/marc/.arduino15/packages/esp32/hardware/esp32/1.0.6/libraries/WiFi wordt gebruikt Bibliotheek Adafruit_LiquidCrystal op versie 1.1.0 in map: /home/marc/Arduino/libraries/Adafruit_LiquidCrystal wordt gebruikt Bibliotheek BMx280MI op versie 1.2.1 in map: /home/marc/Arduino/libraries/BMx280MI wordt gebruikt Bibliotheek ArduinoMqttClient op versie 0.1.5 in map: /home/marc/Arduino/libraries/ArduinoMqttClient wordt gebruikt Bibliotheek SPI op versie 1.0 in map: /home/marc/.arduino15/packages/esp32/hardware/esp32/1.0.6/libraries/SPI wordt gebruikt /home/marc/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-97-gc752ad5-5.2.0/bin/xtensa-esp32-elf-size -A /tmp/arduino_build_49053/BMx280_I2C-20220119-libarduinomqttclient.ino.elf De schets gebruikt 767846 bytes (58%) programma-opslagruimte. Maximum is 1310720 bytes. Globale variabelen gebruiken 41132 bytes (12%) van het dynamisch geheugen. Resteren 286548 bytes voor lokale variabelen. Maximum is 327680 bytes. python /home/marc/.arduino15/packages/esp32/tools/esptool_py/3.0.0/esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 115200 --before default_reset --after hard_res

[lhtrevisan]

Garc964, thank you for your reply, but you are using a different mqtt library(ArduinoMqttClient.h), instead of PubSubClient.

Also, you don't have WiFiClientSecure defined in your code... The code I posted does not use certificates, but for deployment I'll use them, as I intend to use AWS IoT instead of HiveMQ.

In answer to my own question, YES, this memory consumption is absolutely normal, as certificates demand about 5-6kb and pubsub needs another 17-18k.

I'm going for solution number 1 and hoping it does not crash on the long run by lack of memory.