Examples don't work with ESP32-S3 Box 3? (AIS-1455)

[mapigit]

Hi there,

I've tried for several days now to get the Wake-word example working on my new ESP32-S3 Box 3. Different IDF versions gave the same reult: The example compiles without problems (yes, I've set the target to ESP32S3, and in menuconfig the Esp32-S3 Box), but there is no reaction to any wakeword. Same for the en speech recognition. However, the factory demo works fine: I've built and flashed the demo v1.2.4 with IDF 5.1.2; wakeword and english commands are recognized.

Is it possible that the Skainet examples don't work with the Box 3 Hardware version yet? BTW, the LOG of the wakeword example shows MIC1-MIC4, which I find strange, whereas the factory demo correctly prints only MIC1, MIC2.

Can you please help? Thanks, cheers, Martin

Here's the monitor output; after this the box doesn't react to wakewords Hi ESP or Alexa (both selected in menuconfig):

SPIWP:0xee mode:DIO, clock div:1 load:0x3fce3818,len:0x514 load:0x403c9700,len:0x4 load:0x403c9704,len:0xb58 load:0x403cc700,len:0x2b64 entry 0x403c98fc I (24) cpu_start: Multicore app I (25) octal_psram: vendor id : 0x0d (AP) I (26) octal_psram: dev id : 0x03 (generation 4) I (27) octal_psram: density : 0x05 (128 Mbit) I (28) octal_psram: good-die : 0x01 (Pass) I (29) octal_psram: Latency : 0x01 (Fixed) I (30) octal_psram: VCC : 0x00 (1.8V) I (31) octal_psram: SRF : 0x01 (Fast Refresh) I (33) octal_psram: BurstType : 0x01 (Hybrid Wrap) I (35) octal_psram: BurstLen : 0x01 (32 Byte) I (36) octal_psram: Readlatency : 0x02 (10 cycles@Fixed) I (38) octal_psram: DriveStrength: 0x00 (1/1) I (39) MSPI Timing: PSRAM timing tuning index: 5 I (40) esp_psram: Found 16MB PSRAM device I (41) esp_psram: Speed: 80MHz I (42) cpu_start: Pro cpu up. I (43) cpu_start: Starting app cpu, entry point is 0x403793f8 0x403793f8: call_start_cpu1 at C:/Users/Martin/esp/esp-idf/components/esp_system/port/cpu_start.c:157

I (0) cpu_start: App cpu up. I (625) esp_psram: SPI SRAM memory test OK I (634) cpu_start: Pro cpu start user code I (634) cpu_start: cpu freq: 240000000 Hz I (635) cpu_start: Application information: I (636) cpu_start: Project name: wake_word_detection I (637) cpu_start: App version: v0.3.0-304-gba3106e I (638) cpu_start: Compile time: Jan 1 2024 20:37:53 I (640) cpu_start: ELF file SHA256: 8c70aa641326e0b5... I (642) cpu_start: ESP-IDF: v5.1.2 I (643) cpu_start: Min chip rev: v0.0 I (643) cpu_start: Max chip rev: v0.99 I (644) cpu_start: Chip rev: v0.2 I (645) heap_init: Initializing. RAM available for dynamic allocation: I (647) heap_init: At 3FC9CF70 len 0004C7A0 (305 KiB): DRAM I (649) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM I (650) esp_psram: Adding pool of 16384K of PSRAM memory to heap allocator I (652) spi_flash: detected chip: gd I (653) spi_flash: flash io: qio I (653) sleep: Configure to isolate all GPIO pins in sleep state I (655) sleep: Enable automatic switching of GPIO sleep configuration I (657) app_start: Starting scheduler on CPU0 I (658) app_start: Starting scheduler on CPU1 I (658) esp_psram: Reserving pool of 16K of internal memory for DMA/internal allocations I (668) ES7210: Work in Slave mode I (668) ES7210: Enable ES7210_INPUT_MIC1 I (668) ES7210: Enable ES7210_INPUT_MIC2 I (668) ES7210: Enable ES7210_INPUT_MIC3 I (668) ES7210: Enable ES7210_INPUT_MIC4 I (668) ES7210: Enable TDM mode I (668) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3 I (668) I2S_IF: STD Mode 0 bits:32/32 channel:2 sample_rate:16000 mask:3 I (678) ES7210: Bits 16 I (678) ES7210: Enable ES7210_INPUT_MIC1 I (678) ES7210: Enable ES7210_INPUT_MIC2 I (678) ES7210: Enable ES7210_INPUT_MIC3 I (678) ES7210: Enable ES7210_INPUT_MIC4 I (678) ES7210: Enable TDM mode I (688) ES7210: Unmuted I (688) Adev_Codec: Open codec device OK I (688) ES8311: Work in Slave mode I (688) gpio: GPIO[46]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 I (688) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3 I (688) I2S_IF: STD Mode 1 bits:32/32 channel:2 sample_rate:16000 mask:3 I (698) Adev_Codec: Open codec device OK I (698) MODEL_LOADER: The storage free size is 15936 KB I (698) MODEL_LOADER: The partition size is 5168 KB I (698) MODEL_LOADER: Successfully map model partition The first wakenet model: wn9_alexa The second wakenet model: wn9_hiesp I (698) AFE_SR: afe interface for speech recognition

I (698) AFE_SR: AFE version: SR_V220727

I (698) AFE_SR: Initial auido front-end, total channel: 3, mic num: 2, ref num: 1

I (708) AFE_SR: aec_init: 0, se_init: 1, vad_init: 1

I (708) AFE_SR: wakenet_init: 1

MC Quantized wakenet9: wakenet9l_v3h24_alexa_3_0.625_0.645, tigger:v3, mode:2, p:0, (Aug 24 2023 11:56:47) MC Quantized wakenet9: wakeNet9_v1h24_hiesp_3_0.63_0.635, tigger:v3, mode:2, p:0, (Aug 24 2023 11:56:47) I (928) AFE_SR: wake num: 3, mode: 0, (Aug 24 2023 11:56:48)

I (928) main_task: Returned from app_main() ------------detect start------------

[feizi]

@mapigit , Yes, currently, Skainet examples don't work with the Box 3 Hardware version yet. We will support it this week.

[feizi]

Thank you for the question. Now Skainet has supported Box-3. Please try again. The commit: https://github.com/espressif/esp-skainet/commit/8824d838a72b4a16d1cf059fcf677d2c169ffd46

[mapigit]

@feizi, could you please also add "|| CONFIG_ESP32_S3_BOX_3_BOARD" in main.c? E.g. in the english speech recognition example in lines 170 and 154. Thanks!

[mapigit]

Hi there,

if I select "multiple wakewords" in the english speech recognition example and then "Alexa" as well as "Hi, ESP!", only "Alexa" will be recognized as a wakeword by the Box3. If I keep "multiple wakewords", deselect "Alexa" and select only "Hi, ESP!", then "Hi, ESP!" works. Is this a bug? I'd like to be able to use bothe wakewords alternatively in the same program.

Thanks, cheers, Martin

[feizi]

Hi @mapigit ,

if I select "multiple wakewords" in the english speech recognition example and then "Alexa" as well as "Hi, ESP!", only "Alexa" will be recognized as a wakeword by the Box3. If I keep "multiple wakewords", deselect "Alexa" and select only "Hi, ESP!", then "Hi, ESP!" works. Is this a bug? I'd like to be able to use bothe wakewords alternatively in the same program.

It is not a bug. If you keep "multiple wakewords", you also need choose two or more wake words, like this:

[mapigit]

Hi @feizi that's exactly what I did. After selecting "multiple wakewords" as well as both wakewords, still only "Alexa" works. Thanks, cheers, Martin

[sun-xiangyu]

You still need to bind the model name of the WakeNet in the AFE config. Please refer to this example

    afe_config.wakenet_model_name = wn_name;
    afe_config.wakenet_model_name_2 = wn_name_2;