espressif / arduino-esp32

Arduino core for the ESP32
GNU Lesser General Public License v2.1
13.71k stars 7.43k forks source link

coredump corrupted on 3.0.7 #10632

Closed Tarik2142 closed 22 hours ago

Tarik2142 commented 1 day ago

Board

esp32dev

Device Description

-

Hardware Configuration

-

Version

latest master (checkout manually)

IDE Name

pio

Operating System

Windows 10

Flash frequency

-

PSRAM enabled

no

Upload speed

115200

Description

I completely wiped the chip and flashed it from scratch. I didn't get this problem on core 2.x

I get: E (169) esp_core_dump_flash: ����e dump data check failed: Calculated checksum='a1ca788b' Image checksum='ffffffff' after the first assert(0) and coredump doesn't work anymore. Also, the device does not always reboot after a crash, it can remain in halt mode forever.

Sketch

#include <Arduino.h>

void setup() {
    delay(10000);
    assert(0);
}

void loop() {
    delay(1000);
}

Debug Message

// First reboot - OK
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (S0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4604
ho 0 tail 12 room 4
load:0x40078000,len:15468
load:0x40080400,len:4
ho 8 tail 4 room 4
load:0x40080404,len:3180
entry 0x400805b8

assert failed: void setup() main.cpp:5 (0)

Backtrace: 0x40082eb1:0x3ffb20e0 0x400876b9:0x3ffb2100 0x4008c35a:0x3ffb2120 0x400d1627:0x3ffb2250 0x400d17ee:0x3ffb2270 0x400896a1:0x3ffb2290
  #0  0x40082eb1 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
  #1  0x400876b9 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84
  #2  0x4008c35a in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
  #3  0x400d1627 in setup() at src/main.cpp:5
  #4  0x400d17ee in loopTask(void*) at C:/Users/Tarik2142/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:59
  #5  0x400896a1 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

ELF file SHA256: 5520ccd13053a328

Guru Meditation Error: Core  1 panic'ed (Unhandled debug exception). 
Debug exception reason: BREAK instr
Core  1 register dump:
PC      : 0x400837c0  PS      : 0x00000016  A0      : 0x40080306  A1      : 0x3ffc2630
A2      : 0x00000000  A3      : 0x70000005  A4      : 0x00000000  A5      : 0x00000000
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000800  A9      : 0x3ffb1f20
A10     : 0x00000000  A11     : 0x1163e1ea  A12     : 0x1163e1ea  A13     : 0x00060b24
A14     : 0xfffbfff0  A15     : 0x00000046  SAR     : 0x00000000  EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000  LBEG    : 0x40085b88  LEND    : 0x40085b9e  LCOUNT  : 0xffffffff

Backtrace: 0x400837bd:0x3ffc2630 0x40080303:0x3ffc2660 0x65a5a5a2:0xa5a5a5a5 |<-CORRUPTED

ELF file SHA256: 5520ccd13053a328

Re-entered core dump! Exception happened during core dump!
Rebooting...

// All of the following reboots coredump are broken!
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4604
ho 0 tail 12 room 4
load:0x40078000,len:15468
load:0x40080400,len:4
ho 8 tail 4 room 4
load:0x40080404,len:3180
entry 0x400805b8
E (169) esp_core_dump_flash: ����e dump data check failed:
Calculated checksum='a1ca788b'
Image checksum='ffffffff'

assert failed: void setup() main.cpp:5 (0)

Backtrace: 0x40082eb1:0x3ffb20e0 0x400876b9:0x3ffb2100 0x4008c35a:0x3ffb2120 0x400d1627:0x3ffb2250 0x400d17ee:0x3ffb2270 0x400896a1:0x3ffb2290
  #0  0x40082eb1 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
  #1  0x400876b9 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84
  #2  0x4008c35a in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
  #3  0x400d1627 in setup() at src/main.cpp:5
  #4  0x400d17ee in loopTask(void*) at C:/Users/Tarik2142/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:59
  #5  0x400896a1 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

ELF file SHA256: 5520ccd13053a328

Guru Meditation Error: Core  1 panic'ed (Unhandled debug exception). 
Debug exception reason: BREAK instr
Core  1 register dump:
PC      : 0x400837c0  PS      : 0x00000016  A0      : 0x40080306  A1      : 0x3ffc2630
A2      : 0x00000000  A3      : 0x70000005  A4      : 0x00000000  A5      : 0x00000000
A6      : 0x00000000  A7      : 0x00000000  A8      : 0x00000800  A9      : 0x3ffb1f20
A10     : 0x00000000  A11     : 0x81c99f9d  A12     : 0x81c99f9d  A13     : 0x00060b24
A14     : 0xfffbfff0  A15     : 0x00000046  SAR     : 0x00000000  EXCCAUSE: 0x00000001
EXCVADDR: 0x00000000  LBEG    : 0x40085b88  LEND    : 0x40085b9e  LCOUNT  : 0xffffffff

Backtrace: 0x400837bd:0x3ffc2630 0x40080303:0x3ffc2660 0x65a5a5a2:0xa5a5a5a5 |<-CORRUPTED

ELF file SHA256: 5520ccd13053a328

Re-entered core dump! Exception happened during core dump!
Rebooting...

// broken
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4604
ho 0 tail 12 room 4
load:0x40078000,len:15468
load:0x40080400,len:4
ho 8 tail 4 room 4
load:0x40080404,len:3180
entry 0x400805b8
E (169) esp_core_dump_flash: ����e dump data check failed:
Calculated checksum='a1ca788b'
Image checksum='ffffffff'

Other Steps to Reproduce

just crash esp32 :)

I have checked existing issues, online documentation and the Troubleshooting Guide

Tarik2142 commented 23 hours ago

board_upload.flash_size = 16MB board_build.flash_size = 16MB board_build.partitions = default_16MB.csv

me-no-dev commented 23 hours ago

exception does not happen during the coredump error. Your trace clearly says it happened in your assert. Remove the assert and if happens again, we can discuss further. The error can be dismissed as it does not prevent the firmware to work, it just says that coredump data will not be valid if extracted.

Tarik2142 commented 23 hours ago

exception does not happen during the coredump error. Your trace clearly says it happened in your assert. Remove the assert and if happens again, we can discuss further. The error can be dismissed as it does not prevent the firmware to work, it just says that coredump data will not be valid if extracted.

This is the problem, if a crash occurs I can't use coredump to find out the cause because it is corrupted.

me-no-dev commented 23 hours ago

It is generally not enabled on our end for that. It's instead used by RainMaker and ESP-Insights. That is the only reason we have that partition there.

Tarik2142 commented 23 hours ago

Are you sure? I see that it was turned on specifically for crash decoding https://github.com/espressif/arduino-esp32/issues/9808

me-no-dev commented 22 hours ago

I need to doublecheck, but maybe it does not work with assert?

Docs: https://docs.espressif.com/projects/esp-idf/en/v5.1.5/esp32/api-guides/core_dump.html

It mentions cases where coredump might be corrupted.

I didn't get this problem on core 2.x

As far as I remember coredump was off in 2.x

me-no-dev commented 22 hours ago

if a crash occurs I can't use coredump to find out the cause because it is corrupted.

can you show such crash? Because with assert you are given where it happened and also a backtrace along with it

Tarik2142 commented 22 hours ago

if a crash occurs I can't use coredump to find out the cause because it is corrupted.

can you show such crash? Because with assert you are given where it happened and also a backtrace along with it

Here is an example of a dump crashbump-webui-fail.zip

Tarik2142 commented 22 hours ago

In v2.0.6 I could get all the information image but in 3.0.7 he dumps no longer open due to a checksum error

me-no-dev commented 22 hours ago

In that case I will suggest you ask in the ESP-IDF repo if they know about an issue with coredump in IDF 5.1. You could also try to use Arduino 3.1.0-RC3 which is based on IDF 5.3 and see if there is any change. On our end we only enable the feature, but have no control over what it does and how it works. Make sure you have the sdkconfig ready (it's located in the libs package that gets installed with the core)

The example dump you posted is triggered by task WDT. I do not know if that matters.

Jason2866 commented 22 hours ago

A Image checksum='ffffffff' probably there is nothing stored at all.

me-no-dev commented 22 hours ago

yes, but it's clearly enabled to do so in sdkconfig

Tarik2142 commented 22 hours ago

In that case I will suggest you ask in the ESP-IDF repo if they know about an issue with coredump in IDF 5.1. You could also try to use Arduino 3.1.0-RC3 which is based on IDF 5.3 and see if there is any change. On our end we only enable the feature, but have no control over what it does and how it works. Make sure you have the sdkconfig ready (it's located in the libs package that gets installed with the core)

The example dump you posted is triggered by task WDT. I do not know if that matters.

Ok, will try to ask in IDF. I tried 3.1.0-RC3, same problem. Close it for now.