search

espressif / esp-adf

Espressif Audio Development Framework
Other
1.55k stars 685 forks source link

i2s driver crash (SPIRAM optimization applied) (AUD-4481) #971

Closed jw-redpanda closed 1 year ago

jw-redpanda commented 1 year ago

Environment

Core 0 register dump: PC : 0x400865a9 PS : 0x00050034 A0 : 0x40084214 A1 : 0x3ffb0be0 A2 : 0x3f807adc A3 : 0xbad00bad A4 : 0x00000000 A5 : 0x400889ce A6 : 0xe0000020 A7 : 0x00000000 A8 : 0x00000000 A9 : 0x00000000 A10 : 0x3ffb2e70 A11 : 0x00000001 A12 : 0x80084bbd A13 : 0x3ffbecb0 A14 : 0x00000001 A15 : 0x3ffb2bc8 SAR : 0x00000018 EXCCAUSE: 0x00000007 EXCVADDR: 0x00000000 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000002

Backtrace: 0x400865a6:0x3ffb0be0 0x40084211:0x3ffb0c10 0x4000c2e1:0x3ffbeca0 0x4008cf1a:0x3ffbecb0 0x4008fd21:0x3ffbece0 0x400857bf:0x3ffbed60 0x400db826:0x3ffbeda0 0x400 e31a2:0x3ffbedd0 0x400e4391:0x3ffbedf0 0x400e30ba:0x3ffbee30 0x400ddf1d:0x3ffbee50 0x400dde45:0x3ffbee70 0x400de309:0x3ffbee90 0x400ded30:0x3ffbeeb0 0x400dedc9:0x3ffbeed0 0x400df103:0x3ffbef00 0x400e0c4b:0x3ffbef60 0x400d3559:0x3ffbef90 0x400f44ab:0x3ffbefb0 0x400f4550:0x3ffbefe0 0x400d700f:0x3ffbf000 0x400d718b:0x3ffbf030 0x4008c12d:0x3f fbf050

ELF file SHA256: 9fdf0ba3f393bbae

Rebooting...

- Backtrack
```0x400865a6: i2s_ll_get_intr_status at /home/widefly/esp/esp-idf-v4.4.4/components/hal/esp32/include/hal/i2s_ll.h:398
 (inlined by) i2s_intr_handler_default at /home/widefly/esp/esp-idf-v4.4.4/components/driver/i2s.c:435
0x40084211: _xt_medint2 at /home/widefly/esp/esp-idf-v4.4.4/components/freertos/port/xtensa/xtensa_vectors.S:1203
0x4000c2e1: ?? ??:0
0x4008cf1a: spi_flash_ll_get_buffer_data at /home/widefly/esp/esp-idf-v4.4.4/components/hal/esp32/include/hal/spi_flash_ll.h:148
 (inlined by) spi_flash_hal_read at /home/widefly/esp/esp-idf-v4.4.4/components/hal/spi_flash_hal_common.inc:176
0x4008fd21: spi_flash_chip_winbond_read at /home/widefly/esp/esp-idf-v4.4.4/components/spi_flash/spi_flash_chip_winbond.c:69
0x400857bf: esp_flash_read at /home/widefly/esp/esp-idf-v4.4.4/components/spi_flash/esp_flash_api.c:832 (discriminator 4)
0x400db826: esp_partition_read at /home/widefly/esp/esp-idf-v4.4.4/components/spi_flash/partition.c:424
0x00000400: ?? ??:0
0x000e31a2: ?? ??:0
0x400e4391: WL_Flash::read(unsigned int, void*, unsigned int) at /home/widefly/esp/esp-idf-v4.4.4/components/wear_levelling/WL_Flash.cpp:611
0x400e30ba: wl_read at /home/widefly/esp/esp-idf-v4.4.4/components/wear_levelling/wear_levelling.cpp:223
0x400ddf1d: ff_wl_read at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/diskio/diskio_wl.c:46
0x400dde45: ff_disk_read at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/diskio/diskio.c:70
0x400de309: move_window at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/src/ff.c:1077
0x400ded30: dir_find at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/src/ff.c:2450
0x400dedc9: follow_path at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/src/ff.c:3070
0x400df103: f_open at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/src/ff.c:3571
0x400e0c4b: vfs_fat_open at /home/widefly/esp/esp-idf-v4.4.4/components/fatfs/vfs/vfs_fat.c:313
0x400d3559: esp_vfs_open at /home/widefly/esp/esp-idf-v4.4.4/components/vfs/vfs.c:399 (discriminator 3)
0x400f44ab: _fopen_r at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fopen.c:129
0x400f4550: fopen at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdio/fopen.c:168
0x400d700f: isFATFSCorrupted at /home/widefly/projects/m1esp/crashProjects/esp-adf-play_mp3_control/main/play_mp3_control_example.c:174
0x400d718b: worker at /home/widefly/projects/m1esp/crashProjects/esp-adf-play_mp3_control/main/play_mp3_control_example.c:287 (discriminator 15)
0x4008c12d: vPortTaskWrapper at /home/widefly/esp/esp-idf-v4.4.4/components/freertos/port/xtensa/port.c:142
0x00fbf050: ?? ??:0

Problem Description

Expected Behavior

File operations, e.g. read, write, should work when audio is playing or recording.

Actual Behavior

When there is a file operation, e.g. fopen(), it will crash immediately.

Steps to Reproduce

Copied from an esp-idf get-started example, I created a repo which has inserted some test code and is able to reproduce the issue. Please check and see the repo.

The repo inserts code that will run a task to perform file read/write periodically. In details:

Some build assets for your reference

Code to Reproduce This Issue

jw-redpanda commented 1 year ago

I am building a commercial product using esp-idf + esp-adf, which utilizes BLE, Wi-Fi, audio playback/recording, and implemented some parallel running tasks. The memory optimization is critical and required. Since the crash is repeatable and the demo repo code has been prepared for the investigation, including detail backtrace, I look forward to seeing the esp-adf team's response. Thank you.

jason-mao commented 1 year ago

@jw-redpanda Guru Meditation Error: Core 0 panic'ed (Cache disabled but cached memory region accessed). Please refer to fatal errors docs https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html?highlight=crash#fatal-errors

jw-redpanda commented 1 year ago

@jw-redpanda Guru Meditation Error: Core 0 panic'ed (Cache disabled but cached memory region accessed). Please refer to fatal errors docs https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/fatal-errors.html?highlight=crash#fatal-errors

Thanks for your reply.

I am aware of this error. However, this error happens in the i2s driver, instead of the higher level of app code. We, as a developer, has not had enough knowledge to check and fix the drive code.

If the official sample code could crash inside the driver code, is there any possibility that some driver code does not explicitly allocate internal memory, e.g. heap_caps_malloc(memSize, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT), but may use standard malloc() that allocates memory from SPIRAM, when memory optimization is applied? If this is the case, the crash may happen.

TempoTian commented 1 year ago

We have met a similar issue before when turn on maximum malloc() size in bytes, to always put in internal memory and change it too smaller value such as 16. You can enlarge this value to avoid structure which not huge put to psram directly. For inetrrupt can not access psram when cache disabled during fs reading.

jw-redpanda commented 1 year ago

@TempoTian Yes, I guess some developers also encountered a similar issue.

Since this memory optimization (malloc() that uses PSRAM) is officially supported, applying this optimization should not cause a crash at the driver level. Or else the official documentation should indicate this memory optimization should not be applied if some esp-idf/esp-adf components are used.

I still hope that the esp-adf team to look into this issue.

jason-mao commented 1 year ago

Thank you for your advice. I'm going to close this inactive issue. Please reopen this if you have any questions or need any further assistance.