Deallocator problem (probably) on ESP32-WROVER-E

[Szczempus]

Hello guys. Recently, I've been dealing with a persistent error that might indicate a library issue. I have a freeRTOS-based system running on an ESP32-WROVER-E. One of the tasks is responsible for sending serialized data via GSM to an MQTT broker. This is where the problem starts. As I've noticed, exiting the function where a JsonDocument object is created triggers a LoadProhibited error and restarts the ESP. As the backtrace suggests, the library tries to deallocate an empty memory space, which causes the error (Backtrace below). What I have tried is declaring StaticJsonDocument instead of JsonDocument, manually clearing the allocated memory using doc.clear(), and increasing the allocated task stack as well as the buffer size for the message. Unfortunately, none of these actions have worked. What do you think could be the cause of this problem and how can it be resolved? Thanks in advance.

Troubleshooter report
Here is the report generated by the ArduinoJson Troubleshooter:

1. The program uses ArduinoJson 7
2. The issue happens at run time
3. The issue concerns serialization
4. Program crashes
5. Program doesn't use PROGMEM
6. Program doesn't insert string pointers into the JsonDocument
7. Output type is char[]
8. Switching to String doesn't solve the issue

Environment
Here is the environment that I'm using':

• Microconroller: ESP32-WROVER-E
• Core/runtime: PlatformIO, Arduino framework, espressif32 6.6.0 platform, ArduinoJSON 7.1.0
• IDE: VS Code

Reproduction
Here is a small snippet that demonstrate the problem.

void ModemConnectivityTask(void*){
...
   // Check mqtt connection
    if (!mqttManager.mqttClient.connected()){
      mqttManager.reconnect();
    }

    // Get local timestamp and send data
    networkManager.getTimeStamp(&timeInfo);
    sendSensors();
    sendHealing();
    sendSystem();

    // Update if new message received
    mqttManager.mqttClient.loop();
...
}

void sendHealing(){

  log_i("Before doc");
  JsonDocument doc;
  log_i("After doc");
  char time_buff[64];
  strftime(time_buff, sizeof(time_buff), "%Y.%B.%d-%H:%M:%S", &timeInfo);
  doc["ID"] = DEVICE_ID;
  doc["timestamp"] = String(time_buff);
  doc["status"] = healing_status;

  String jsonString;
  uint8_t len = serializeJsonPretty(doc, jsonString);
  String topic = "/hive/" + String(DEVICE_ID) + "/healing_status";
  log_i("Before sending");
  mqttManager.mqttClient.publish(topic.c_str(), jsonString.c_str(), len);
  log_i("After sending");
  log_i("Memorey overflowed: %d", doc.overflowed());

}

Program output
If relevant, include the program output.

Actual output:

[ 38288][I][main.cpp:934] sendSensors(): Before sending
[ 38436][I][main.cpp:936] sendSensors(): After sending
[ 38441][I][main.cpp:937] sendSensors(): Memorey overflowed: 0
[ 38447][I][main.cpp:977] sendHealing(): Before doc
[ 38453][I][main.cpp:979] sendHealing(): After doc
[ 38461][I][main.cpp:991] sendHealing(): Before sending
[ 38581][I][main.cpp:993] sendHealing(): After sending
[ 38586][I][main.cpp:994] sendHealing(): Memorey overflowed: 0
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x400d29de  PS      : 0x00060730  A0      : 0x800d6245  A1      : 0x3ffc8090  
A2      : 0x3ffc80d0  A3      : 0x3ffbdb84  A4      : 0x3f400124  A5      : 0x000003e2
A6      : 0x3f401018  A7      : 0x00000000  A8      : 0x800d29e8  A9      : 0x3ffc8070
A10     : 0x000081c1  A11     : 0x3ffbdb84  A12     : 0x3ff40000  A13     : 0x00000004  
A14     : 0x3ff50000  A15     : 0xff000000  SAR     : 0x0000001d  EXCCAUSE: 0x0000001c
EXCVADDR: 0x000081c1  LBEG    : 0x40089e5d  LEND    : 0x40089e6d  LCOUNT  : 0xfffffffc  

Backtrace: 0x400d29db:0x3ffc8090 0x400d6242:0x3ffc80b0 0x400d7010:0x3ffc81c0
  #0  0x400d29db in ArduinoJson::V710PB22::detail::StringPool::clear(ArduinoJson::V710PB22::Allocator*) at .pio/libdeps/debug_gms/ArduinoJson/src/ArduinoJson/Memory/StringPool.hpp:33
      (inlined by) ArduinoJson::V710PB22::detail::ResourceManager::~ResourceManager() at .pio/libdeps/debug_gms/ArduinoJson/src/ArduinoJson/Memory/ResourceManager.hpp:25
  #1  0x400d6242 in ArduinoJson::V710PB22::JsonDocument::~JsonDocument() at .pio/libdeps/debug_gms/ArduinoJson/src/ArduinoJson/Document/JsonDocument.hpp:20
      (inlined by) sendHealing() at src/main.cpp:978
  #2  0x400d7010 in ModemConnectivityTask(void*) at src/main.cpp:782

[bblanchon]

Hi @Szczempus,

This looks like a stack corruption, but I cannot see the reason on the sample code. Can you provide an MCVE that doesn't rely on MQTT, strftime, and FreeRTOS?

Best regards, Benoit

[Szczempus]

While working on the proper code example, I found that this problem is likely related to FreeRTOS memory allocation. When I ran the sample code for the first time in the default loop (with all functions executed sequentially), everything ran smoothly for hours without any issues. The second time, I ran the code within a single FreeRTOS task, which also worked without errors. However, when I separated one part of my code to read values from sensors and another to read values from the structure and send them via MQTT in different tasks, the errors reappeared. I use mutexes to access variables and structures, so there shouldn't be any race conditions. I suspect that using heap allocation through FreeRTOS functions (pvPortMalloc()/vPortFree()) might resolve the issue? I will give it a try.

[bblanchon]

I'm not familiar with FreeRTOS, but using pvPortMalloc() and vPortFree() sounds like a good idea.

ArduinoJson allows replacing the allocator by implementing the ArduinoJson::Allocator interface and passing an instance to JsonDocument's constructor. See: https://arduinojson.org/v7/api/jsondocument/#allocator

However, ArduinoJson::Allocator::reallocate() requires a realloc()-like behavior, which unfortunately is not built-in FreeRTOS (there is no pvPortRealloc()), so you'll have to implement the functionality from pvPortMalloc() and vPortFree().