Memory issues for very large program

[shofixti244]

Description:

Hey everyone, please do forgive me if I've come to the wrong neighborhood with this question, but I've been struggling with it for a few days now and I'm pretty much stumped.

So I've been working with the ESP32 (Adafruit HUZZAH32 to be exact) for quite some time now, working on a basic IoT project that involves connecting to Amazon Web Services. I've managed to get Wifi, BLE and the AWS connection all working in separate projects, but now that I'm bringing them together I'm hitting some issues.

After integrating my BLE library, my ported AWS library, and my Wifi code, I was surprised when all of a sudden my serial output started to look like this:

And of course, asides from the garbage Serial output, the program stopped functioning in general.

2 days of trial and error later, I find that so long as the final compressed size of my codebase is larger than 750,000 bytes, I encounter this problem.

Not being an experienced embedded programmer myself, the only conclusion that I've been able to arrive at so far is that the programs code is so large that I'm seeing a memory over-run issue. So I turned my attention to the Partitions table of the board, and made some modifications:

If I'm understanding this all correctly, this should allow for more than enough memory for my program to operate. However I'm still seeing the garbage output and lack of functionality mentioned above.

All the details about my setup are described below... Any input at all here would be a fantastic help. I'm only just learning this whole new world of development, so it may be something really obvious that I'm just not aware of.

Setup:

Board: Adafruit HUZZAH32 IDE name: Atom IDE, Platform.io, Espressif 32 (Stage) Libraries: awsiot for embedded C (ported), ESP32 BLE Arduino arduino-esp32 commit: aba1d7d

[me-no-dev]

Hi :) So shorter sketches with the same configuration work? Can you post a minimal sketch that shows the problem?

[shofixti244]

I'm out of the office for a bit, but I'll attempt to make a simplified repeatable sketch on Sunday. I'll need to find a way to write a program that uses 750,00 bytes in a "simple" way, but I'm sure I can figure something out.

Also, Thank you for the assistance... been stressing abo

[everslick]

I bet my life that you have a memory corruption bug in your sketch that only reveals itself, because your changes also alter the memory layout of your binary. especially the garbled output hints on a Serial.print() with a stray pointer. a static code analyses tool like cppcheck might help.

My binary is currently a tiny bit short of 900K without those issues. (i got different ones, not related to size though) ;-)

Linking ../build/ESP32/3.0.0/alpha/ger/genesys.bin
  Versions: 2.4.0-252-g571a7a5-dirty, 80a7d60
esptool.py v2.1

Memory usage
  Ram:    45104 bytes
  Flash: 892191 bytes

Flash size: 4MB (32Mb)

[shofixti244]

2 more days of debugging have come up with nothing here. I tried out CPPCheck and resolved all of the warnings it produced, however I've still got this issue.

I CAN confirm that this is not simply a code size issue. It seems it's just coincidence that things started to go awry past 700,000 bytes. I'm trying to figure out how to best proceed under everslick's theory that it is some sort of memory corruption bug, but since I can confirm that my issue appears to be about my code, and is unrelated to the ESP32 environment or setup, I'd understand if closing this thread was appropriate.

[everslick]

This issue is closed, because it looks as if it is not a bug or problem with the ESP32 Arduino core or its support libraries. For general API usage questions or help on specific coding challenges, please visit the arduino-esp32 Gitter channel. If you feel this issue was closed in error, reopen it and comment, why you think this is a bug in the Arduino-Core.