BLE crash: Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled. (IDFGH-2935)

joehui commented 4 years ago

I have been getting these BLE related crashes as below. Sometimes the crash would go away if I change some menuconfig setting such as various stack sizes.

I (5927) BleClient: Stop scan successfully
I (5931) BleClient: Unhandled GAP event 23
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x4000c26c  PS      : 0x00060d30  A0      : 0x80122558  A1      : 0x3ffd8a90  
A2      : 0x81000000  A3      : 0x3ffe0ff5  A4      : 0x00000006  A5      : 0x3ffc22a8  
A6      : 0x00000000  A7      : 0x3ffd36dc  A8      : 0x00000001  A9      : 0x81000000  
A10     : 0x00000000  A11     : 0x3ffe0e14  A12     : 0x3ffba4a8  A13     : 0x3ffba4a8  
A14     : 0x00000001  A15     : 0x3ffd76ac  SAR     : 0x00000018  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x81000000  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0x00000000  

ELF file SHA256: 450e1c6effe3e1ddaaa0ea71fec63041f6c2af090bf7fa9d71edbabf69accdfb

Backtrace: 0x4000c269:0x3ffd8a90 |<-CORRUPTED

Entering gdb stub now.
$T0b#e6GNU gdb (crosstool-NG esp-2019r2) 8.1.0.20180627-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-host_apple-darwin12 --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /Users/jhui/esp32/smartswitchesp32/build/smartswitchesp32.elf...done.
Remote debugging using /dev/cu.SLAB_USBtoUART
0x4000c26c in ?? ()
(gdb) bt
#0  0x4000c26c in ?? ()
#1  0x40122558 in bta_gattc_co_cache_find_src_addr (assoc_addr=0x3ffe0ff5 "\354\214u_\327`\001\001", index=0x3ffd8c4b "\377\211\241\033\217\250\"\374?\034\016\376?\003")
    at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_co.c:630
#2  0x4012261d in cacheOpen (bda=0x3ffe0ff5 "\354\214u_\327`\001\001", index=0x3ffd8c4b "\377\211\241\033\217\250\"\374?\034\016\376?\003", to_save=<optimized out>)
    at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_co.c:134
#3  0x401226b8 in bta_gattc_co_cache_open (server_bda=0x3ffe0ff5 "\354\214u_\327`\001\001", to_save=true, index=0x3ffd8c4b "\377\211\241\033\217\250\"\374?\034\016\376?\003")
    at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_co.c:239
#4  0x40121e14 in bta_gattc_cache_load (p_clcb=0x3ffe0e14) at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_cache.c:2127
#5  0x401334e5 in bta_gattc_conn (p_clcb=0x3ffe0e14, p_data=0x3ffba478) at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_act.c:674
#6  0x401229c1 in bta_gattc_sm_execute (p_clcb=0x3ffe0e14, event=<optimized out>, p_data=0x3ffba478) at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_main.c:292
#7  0x40122acc in bta_gattc_hdl_event (p_msg=0x3ffba478) at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/gatt/bta_gattc_main.c:404
#8  0x40123bc8 in bta_sys_event (param=0x3ffba478) at /users/jhui/esp32/esp-idf/components/bt/host/bluedroid/bta/sys/bta_sys_main.c:499
#9  0x4011ebaa in osi_thread_run (arg=<optimized out>) at /users/jhui/esp32/esp-idf/components/bt/common/osi/thread.c:68
(gdb)

The crash doesn't seem to be related to what the firmware is doing at that moment. It seems like something it is doing in the background. Any idea what the cause it and how to fix it?

I am using IDF 4.0.

Thanks, Joseph

Alvin1Zhang commented 4 years ago

@joehui Thanks for reporting. Would you please help provide more details as suggested in the issue template? Information like elf, sdk configuration, backtrace, log outputs, commit ID, hardware and etc. would help us debug further. Thanks.

joehui commented 4 years ago

Environment

Development Kit: ESP32-DevKit
Kit version (for WroverKit/PicoKit/DevKitC): v1
Module or chip used: ESP32-WROOM-32
IDF version (run git describe --tags to find it): v4.0
Build System: CMake|idf.py
Compiler version (run xtensa-esp32-elf-gcc --version to find it): xtensa-esp32-elf-gcc (crosstool-NG esp-2019r2) 8.2.0
Operating System: macOS
Using an IDE?: No
Power Supply: USB

Problem Description

I have been getting these BLE related crashes as below. Sometimes the crash would go away if I change some menuconfig setting such as various bluetooth stack sizes. The crash locations are kind of random.

Expected Behavior

I should be able to keep the firmware running without crash.

Actual Behavior

Crash in the middle of running at some random non BLE related places.

Steps to reproduce

Just start the firmware.
Crash happens at random place before wifi even finish connecting.

Code to reproduce this issue

Debug Logs

See initial message.

Other items if possible

I noticed that if I change these Stack sizes in menuconfig, the problem seems to be go away CONFIG_BT_BTC_TASK_STACK_SIZE=6572 CONFIG_BT_BTU_TASK_STACK_SIZE=8096

The original values were: CONFIG_BT_BTC_TASK_STACK_SIZE=3572 CONFIG_BT_BTU_TASK_STACK_SIZE=5096

The old values used to work. Nothing related to BLE was changed, and it suddenly stopped working. I wonder if by changing these stack sizes, I have only hidden the problem away. What are the normal expected values for these stack sizes?

I am also attaching the sdkconfig.

sdkconfig.txt

Alvin1Zhang commented 4 years ago

@joehui Thanks for your additional information, we will look into. Thanks.

WCCWCC commented 4 years ago

Hi, @joehui

I see that you have a connection to wifi. I cannot reproduce the problem from your description for the time being.
In order to reproduce this problem, can you provide the project file.
Maybe you can provide a complete project, we will help you solve the problem as soon as possible.

joehui commented 4 years ago

The project I have is quite big thus it is hard to provide you the project.

In our project, we use BluFi for wifi credentials configuration, and the ESP32 also connects to BLE IoT devices. This means the ESP32 acts as both the server and client. I am wondering how much of stack sizes would you recommend one to allocate for Bluedroid in such use case.

Thanks, Joseph

WCCWCC commented 4 years ago

Hi @joehui , I have a method, you can use the freertos API ( uxTaskGetStackHighWaterMark ) to query the minimum value of the stack of the current task. Then adjust further according to this value.

https://www.freertos.org/wp-content/uploads/2018/07/FreeRTOS_Reference_Manual_V10.0.0.pdf

espressif / esp-idf