I2S Port is In Use - wwe example (AUD-5103)

NickEngmann commented 6 months ago

Environment

Audio development kit: ESP32-S3-Korvo-2
Audio kit version: v3
[Required] Module or chip used: ESP32-S3-WROOM-1
[Required] IDF version (run git describe --tags in $IDF_PATH folder to find it): v5.1.2
[Required] ADF version (run git describe --tags in $ADF_PATH folder to find it): v2.6-44-g3c06f0a5
Build system: CMake and the IDF Visual Studio Code Extension
[Required] Running log: All logs from power-on to problem recurrence Microsoft Windows [Version 10.0.23606.1000] (c) Microsoft Corporation. All rights reserved.

C:\Users\nicke\esp\esp-adf\examples\speech_recognition\wwe>set IDF_PATH=C:\Users\nicke\esp\esp-idf

C:\Users\nicke\esp\esp-adf\examples\speech_recognition\wwe>C:\Users\nicke.espressif\python_env\idf5.1_py3.12_env\Scripts\python.exe C:\Users\nicke\esp\esp-idf\tools\idf_monitor.py -p COM6 -b 115200 --toolchain-prefix xtensa-esp32s3-elf- --target esp32s3 c:\Users\nicke\esp\esp-adf\examples\speech_recognition\wwe\build\example_wwe.elf --- WARNING: GDB cannot open serial ports accessed as COMx --- Using \.\COM6 instead... --- esp-idf-monitor 1.3.4 on \.\COM6 115200 --- --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H --- ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x1 (POWERON),boot:0x29 (SPI_FAST_FLASH_BOOT) SPIWP:0xee mode:DIO, clock div:1 load:0x3fce3818,len:0x18e4 load:0x403c9700,len:0x4 load:0x403c9704,len:0xda4 load:0x403cc700,len:0x3084 entry 0x403c9934 I (27) boot: ESP-IDF v5.1.2-dirty 2nd stage bootloader I (27) boot: compile time Dec 24 2023 11:59:19 I (27) boot: Multicore bootloader I (30) boot: chip revision: v0.1 I (34) qio_mode: Enabling default flash chip QIO I (40) boot.esp32s3: Boot SPI Speed : 80MHz I (44) boot.esp32s3: SPI Mode : QIO I (49) boot.esp32s3: SPI Flash Size : 8MB I (54) boot: Enabling RNG early entropy source... I (59) boot: Partition Table: I (63) boot: ## Label Usage Type ST Offset Length I (70) boot: 0 nvs WiFi data 01 02 00009000 00004000 I (78) boot: 1 otadata OTA data 01 00 0000d000 00002000 I (85) boot: 2 phy_init RF data 01 01 0000f000 00001000 I (92) boot: 3 ota_0 OTA app 00 10 00010000 00290000 I (100) boot: 4 model Unknown data 01 82 002a0000 00500000 I (107) boot: 5 flash_tone Unknown data 01 27 007a0000 00032000 I (115) boot: End of partition table I (119) esp_image: segment 0: paddr=00010020 vaddr=3c0a0020 size=32cech (208108) map I (159) esp_image: segment 1: paddr=00042d14 vaddr=3fc9b300 size=05500h ( 21760) load I (164) esp_image: segment 2: paddr=0004821c vaddr=40378000 size=07dfch ( 32252) load I (172) esp_image: segment 3: paddr=00050020 vaddr=42000020 size=95368h (611176) map I (266) esp_image: segment 4: paddr=000e5390 vaddr=4037fdfc size=0b490h ( 46224) load I (284) boot: Loaded app from partition at offset 0x10000 I (284) boot: Disabling RNG early entropy source... I (295) cpu_start: Multicore app I (296) octal_psram: vendor id : 0x0d (AP) I (296) octal_psram: dev id : 0x02 (generation 3) I (299) octal_psram: density : 0x03 (64 Mbit) I (304) octal_psram: good-die : 0x01 (Pass) I (310) octal_psram: Latency : 0x01 (Fixed) I (315) octal_psram: VCC : 0x01 (3V) I (320) octal_psram: SRF : 0x01 (Fast Refresh) I (326) octal_psram: BurstType : 0x01 (Hybrid Wrap) I (332) octal_psram: BurstLen : 0x01 (32 Byte) I (337) octal_psram: Readlatency : 0x02 (10 cycles@Fixed) I (343) octal_psram: DriveStrength: 0x00 (1/1) I (349) MSPI Timing: PSRAM timing tuning index: 5 I (354) esp_psram: Found 8MB PSRAM device I (359) esp_psram: Speed: 80MHz I (362) cpu_start: Pro cpu up. I (366) cpu_start: Starting app cpu, entry point is 0x4037971c 0x4037971c: call_start_cpu1 at C:/Users/nicke/esp/esp-idf/components/esp_system/port/cpu_start.c:157

I (0) cpu_start: App cpu up. I (658) esp_psram: SPI SRAM memory test OK I (667) cpu_start: Pro cpu start user code I (667) cpu_start: cpu freq: 240000000 Hz I (667) cpu_start: Application information: I (670) cpu_start: Project name: example_wwe I (675) cpu_start: App version: v2.6-44-g3c06f0a5-dirty I (681) cpu_start: Compile time: Dec 24 2023 12:10:55 I (688) cpu_start: ELF file SHA256: 2a98203ac0bc0f3a... I (694) cpu_start: ESP-IDF: v5.1.2-dirty I (699) cpu_start: Min chip rev: v0.0 I (704) cpu_start: Max chip rev: v0.99 I (708) cpu_start: Chip rev: v0.1 I (713) heap_init: Initializing. RAM available for dynamic allocation: I (720) heap_init: At 3FCA1878 len 00047E98 (287 KiB): DRAM I (727) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM I (733) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM I (740) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator I (747) spi_flash: detected chip: gd I (751) spi_flash: flash io: qio W (755) spi_flash: Detected size(16384k) larger than the size in the binary image header(8192k). Using the size in the binary image header. W (768) i2s(legacy): legacy i2s driver is deprecated, please migrate to use driver/i2s_std.h, driver/i2s_pdm.h or driver/i2s_tdm.h W (781) ADC: legacy driver is deprecated, please migrate to esp_adc/adc_oneshot.h I (789) sleep: Configure to isolate all GPIO pins in sleep state I (796) sleep: Enable automatic switching of GPIO sleep configuration I (803) app_start: Starting scheduler on CPU0 I (808) app_start: Starting scheduler on CPU1 I (808) main_task: Started on CPU0 I (818) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations I (828) main_task: Calling app_main() I (828) SDCARD: Using 1-line SD mode, base path=/sdcard I (838) gpio: GPIO[15]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (848) gpio: GPIO[7]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (858) gpio: GPIO[4]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (918) SDCARD: CID name SL16G!

I (1368) DRV8311: ES8311 in Slave mode I (1378) gpio: GPIO[48]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 I (1388) ES7210: ES7210 in Slave mode I (1398) ES7210: Enable ES7210_INPUT_MIC1 I (1398) ES7210: Enable ES7210_INPUT_MIC2 I (1398) ES7210: Enable ES7210_INPUT_MIC3 W (1408) ES7210: Enable TDM mode. ES7210_SDP_INTERFACE2_REG12: 2 I (1408) ES7210: config fmt 60 W (1418) AUDIO_BOARD: The board has already been initialized!

----------------------------- ESP Audio Platform ----------------------------- | | | ESP_AUDIO-v1.7.3-7e1b14a-d08db1d | | Compile date: Jul 31 2023-03:46:33 |

I (1458) wwe: Func:setup_player, Line:142, MEM Total:8607052 Bytes, Inter:276451 Bytes, Dram:276451 Bytes

I (1458) wwe: esp_audio instance is:0x3c0e8664

E (1468) i2s(legacy): i2s_check_cfg_validity(926): this i2s port is in use E (1478) i2s(legacy): i2s_driver_install(1582): I2S configuration is invalid I (1488) wwe: Recorder has been created I (1488) MODEL_LOADER: The storage free size is 23680 KB I (1488) MODEL_LOADER: The partition size is 5120 KB I (1498) MODEL_LOADER: Successfully map model partition I (1508) RECORDER_SR: The first wakenet model: wn9_hiesp

I (1508) AFE_SR: afe interface for speech recognition

I (1518) AFE_SR: AFE version: SR_V220727

I (1518) AFE_SR: Initial auido front-end, total channel: 3, mic num: 2, ref num: 1

I (1528) AFE_SR: aec_init: 1, se_init: 1, vad_init: 1

I (1538) AFE_SR: wakenet_init: 1

MC Quantized wakenet9: wakeNet9_v1h24_hiesp_3_0.63_0.635, tigger:v3, mode:2, p:0, (Sep 13 2023 17:58:02) I (1788) AFE_SR: wake num: 3, mode: 0, (Sep 13 2023 18:00:41)

E (1788) RECORDER_SR: recorder_sr.c:598 (recorder_sr_create): Got NULL Pointer I (1798) AFE_SR: afe_se_task quit

I (1798) AFE_SR: afe_mase task destroy finished

I (1808) AFE_SR: exit function: afe_destory

I (1808) main_task: Returned from app_main(

Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it): // 1.22.0-80-g6c4433a
Operating system: Windows
(Windows only) Environment type: ESP Command Prompt
Using an IDE?: Yes, Visual Studio Code
Power supply: USB

Problem Description

E (1468) i2s(legacy): i2s_check_cfg_validity(926): this i2s port is in use E (1478) i2s(legacy): i2s_driver_install(1582): I2S configuration is invalid

After getting that error the recorder doesn't seem to properly initialize, because the wake word is not working

Expected Behavior

Wake word initialization

Actual Behavior

No wake word initialization

Steps to Reproduce

Clone the repo, and build and flash the WWE example with the IDF version and ADF version I mentioned above

[x] sdkconfig file (Attach the sdkconfig file from your project folder) https://gist.github.com/NickEngmann/c7a6dca6a9c76168e975bbfb90e428ea

BootsSiR commented 5 months ago

@NickEngmann any luck getting this to work? I'm encountering the same issue

jason-mao commented 5 months ago

@NickEngmann

E (1468) i2s(legacy): i2s_check_cfg_validity(926): this i2s port is in use E (1478) i2s(legacy): i2s_driver_install(1582): I2S configuration is invalid

This is due to the I2S stream is not fully support the new I2S driver. Please test with a32e66e4d7d6ead511c9be7e8adfd69bc56ab3c7, which is new I2S stream for I2S v5.x driver.

The fatal error is RECORDER_SR: recorder_sr.c:598 (recorder_sr_create): Got NULL Pointer, but I can't reproduce your issue with your mentioned version. Could you help to test with the latest ADF version, and check if it still has the same issue?

lonesix commented 3 months ago

I (12240) AFE_SR: aec_init: 0, se_init: 1, vad_init: 1

I (12240) AFE_SR: wakenet_init: 1

Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled.

Core 0 register dump: PC : 0x4201571f PS : 0x00060130 A0 : 0x82013058 A1 : 0x3fcf38e0 0x4201571f: esp_bss_hps16fft_init at /home/sunxiangyu/workspace/esp_sr_lib/build/../components/esp_audio_processor/microphone_array_speech_enhancement/esp_bss_hps16fft.c:50

A2 : 0x00000040 A3 : 0x3f000000 A4 : 0x00003e80 A5 : 0x00002fd0 A6 : 0x00000000 A7 : 0x00000001 A8 : 0x3f800000 A9 : 0x3fcf38c0 A10 : 0x00000000 A11 : 0x00000004 A12 : 0x00000404 A13 : 0x3fcf38f0 A14 : 0x3fcf38d0 A15 : 0x0000000c SAR : 0x00000004 EXCCAUSE: 0x0000001d EXCVADDR: 0x00000020 LBEG : 0x400556d5 LEND : 0x400556e5 LCOUNT : 0xfffffffe

Backtrace: 0x4201571c:0x3fcf38e0 0x42013055:0x3fcf3910 0x4200d937:0x3fcf3940 0x42008a06:0x3fcf3980 0x42008a46:0x3fcf3b40 0x42008895:0x3fcf3b60 0x42008553:0x3fcf3b80 0x420cbf04:0x3fcf3bb0 0x403810b1:0x3fcf3bd0 0x4201571c: esp_bss_hps16fft_init at /home/sunxiangyu/workspace/esp_sr_lib/build/../components/esp_audio_processor/microphone_array_speech_enhancement/esp_bss_hps16fft.c:51

0x42013055: afe_create_from_config at /home/sunxiangyu/workspace/esp_sr_lib/build/../components/esp_audio_front_end/esp_afe_sr.c:281

0x4200d937: recorder_sr_create at D:/esp-idf/ESP/esp-adf2/components/audio_recorder/recorder_sr.c:573

0x42008a06: duer_audio_start_recorder at D:/esp-idf/project/http/sample_project/main/mic_speaker.c:225

0x42008a46: mic_speaker_init at D:/esp-idf/project/http/sample_project/main/mic_speaker.c:288

0x42008895: audio_app_init at D:/esp-idf/project/http/sample_project/main/https_request.c:237

0x42008553: app_main at D:/esp-idf/project/http/sample_project/main/main.c:63 (discriminator 13)

0x420cbf04: main_task at D:/esp-idf/v4.4.6/esp-idf/components/freertos/port/port_common.c:141 (discriminator 2)

0x403810b1: vPortTaskWrapper at D:/esp-idf/v4.4.6/esp-idf/components/freertos/port/xtensa/port.c:142

I am the same development board, IDF4.4.6 First,I don't know if this link location works. 0x4201571c: esp_bss_hps16fft_init at /home/sunxiangyu/workspace/esp_sr_lib/build/../components/esp_audio_processor/microphone_array_speech_enhancement/esp_bss_hps16fft.c:51 0x42013055: afe_create_from_config at /home/sunxiangyu/workspace/esp_sr_lib/build/../components/esp_audio_front_end/esp_afe_sr.c:281

Second, when you run here, a serious error occurs: recorder_sr_cfg_t recorder_sr_cfg = DEFAULT_RECORDER_SR_CFG(); recorder_sr_cfg.afe_cfg.aec_init = false; recorder_sr_cfg.afe_cfg.memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM; recorder_sr_cfg.afe_cfg.agc_mode = AFE_MN_PEAK_NO_AGC;

audio_rec_cfg_t cfg = AUDIO_RECORDER_DEFAULT_CFG();
cfg.read = (recorder_data_read_t)&input_cb_for_afe;
cfg.sr_handle = recorder_sr_create(&recorder_sr_cfg, &cfg.sr_iface);/***Here***/
cfg.event_cb = cb;
cfg.vad_off = 1000;
recorder = audio_recorder_create(&cfg);

I sincerely hope that someone can answer my questions, thank you very much!!!

espressif / esp-adf