search

espressif / esp-adf

Espressif Audio Development Framework
Other
1.55k stars 687 forks source link

WWE example with LyraT board 4.3 issue (AUD-3824) #822

Closed chaitanya-smartrotamac closed 2 years ago

chaitanya-smartrotamac commented 2 years ago

I tried ADF example wwe with my LyraT 4.3 board with IDF:4.4 ADF2.4 I found following errors from lyra serial 

E (1384) gpio: gpio_install_isr_service(449): GPIO isr service already installed
W (1384) PERIPH_TOUCH: _touch_init
W (1404) AUDIO_BOARD: The board has already been initialized!
E (1464) I2S: register I2S object to platform failed
Deprecated Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0 (Dec 10 2021 20:15:15)
3
Initial ONE-MIC auido front-end for speech recognition, mode:1, (Dec 10 2021 20:29:24)
SINGLE_RECOGNITION: v2; (Dec 10 2021 20:45:08)
W (1724) MN: MultiNet may cause WatchDog on ESP32 sometimes!

:- Extra point done apart from guide given is enabling the Multinet in menuconfig because it gave error as Multinet is not enabled.

Nothing happening after thus warning.

Is something wrong I am doing? Is this example compatible with LyraT board?

chaitanya-smartrotamac commented 2 years ago

Complete log file

rst:0x1 (POWERON_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:6972
ho 0 tail 12 room 4
load:0x40078000,len:14432
load:0x40080400,len:4424
0x40080400: _init at ??:?

entry 0x400806ac
I (29) boot: ESP-IDF v4.4-329-g3c9657fa4e-dirty 2nd stage bootloader
I (29) boot: compile time 11:52:02
I (29) boot: chip revision: 3
I (34) qio_mode: Enabling default flash chip QIO
I (39) boot.esp32: SPI Speed      : 80MHz
I (44) boot.esp32: SPI Mode       : QIO
I (48) boot.esp32: SPI Flash Size : 4MB
I (53) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (62) boot: ## Label            Usage          Type ST Offset   Length
I (69) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (76) boot:  1 otadata          OTA data         01 00 0000d000 00002000
I (84) boot:  2 phy_init         RF data          01 01 0000f000 00001000
I (91) boot:  3 ota_0            OTA app          00 10 00010000 00300000
I (99) boot:  4 flash_tone       Unknown data     01 27 00310000 00032000
I (106) boot: End of partition table
I (111) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=177710h (1537808) map
I (543) esp_image: segment 1: paddr=00187738 vaddr=3ffb0000 size=02468h (  9320) load
I (546) esp_image: segment 2: paddr=00189ba8 vaddr=40080000 size=06470h ( 25712) load
I (557) esp_image: segment 3: paddr=00190020 vaddr=400d0020 size=72048h (467016) map
I (686) esp_image: segment 4: paddr=00202070 vaddr=40086470 size=09690h ( 38544) load
I (699) esp_image: segment 5: paddr=0020b708 vaddr=50000000 size=00010h (    16) load
I (707) boot: Loaded app from partition at offset 0x10000
I (707) boot: Disabling RNG early entropy source...
I (719) psram: This chip is ESP32-D0WD
I (719) spiram: Found 64MBit SPI RAM device
I (719) spiram: SPI RAM mode: flash 80m sram 80m
I (722) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (730) cpu_start: Pro cpu up.
I (733) cpu_start: Starting app cpu, entry point is 0x4008153c
0x4008153c: call_start_cpu1 at C:/Espressif/frameworks/esp-adf-v2.4/esp-idf/components/esp_system/port/cpu_start.c:160

I (0) cpu_start: App cpu up.
I (1215) spiram: SPI SRAM memory test OK
I (1222) cpu_start: Pro cpu start user code
I (1222) cpu_start: cpu freq: 240000000
I (1223) cpu_start: Application information:
I (1226) cpu_start: Project name:     example_wwe
I (1231) cpu_start: App version:      1
I (1235) cpu_start: Compile time:     May 17 2022 11:51:49
I (1242) cpu_start: ELF file SHA256:  658115f561aa4ef4...
I (1248) cpu_start: ESP-IDF:          v4.4-329-g3c9657fa4e-dirty
I (1254) heap_init: Initializing. RAM available for dynamic allocation:
I (1262) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (1268) heap_init: At 3FFB44B8 len 0002BB48 (174 KiB): DRAM
I (1274) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1281) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1287) heap_init: At 4008FB00 len 00010500 (65 KiB): IRAM
I (1293) spiram: Adding pool of 4095K of external SPI memory to heap allocator
I (1302) spi_flash: detected chip: generic
I (1306) spi_flash: flash io: qio
W (1310) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
I (1324) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (1334) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1344) gpio: GPIO[36]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1354) gpio: GPIO[39]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1364) gpio: GPIO[19]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
E (1384) gpio: gpio_install_isr_service(449): GPIO isr service already installed
W (1384) PERIPH_TOUCH: _touch_init
I (1394) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1394) ES8388_DRIVER: init,out:02, in:00
W (1404) AUDIO_BOARD: The board has already been initialized!

----------------------------- ESP Audio Platform -----------------------------
|                                                                            |
|                 ESP_AUDIO-v1.7.0-63-g2b9ee8e-bec4a56-4d51bc8               |
|                     Compile date: Feb 23 2022-02:43:28                     |
------------------------------------------------------------------------------
I (1444) LYRAT_V4_3: I2S0, MCLK output by GPIO0
I (1454) wwe_example: Func:setup_player, Line:131, MEM Total:4427551 Bytes, Inter:333847 Bytes, Dram:269183 Bytes

I (1464) wwe_example: esp_audio instance is:0x3f801b44

E (1464) I2S: register I2S object to platform failed
I (1484) LYRAT_V4_3: I2S0, MCLK output by GPIO0
I (1484) wwe_example: Recorder has been created
Deprecated Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0 (Dec 10 2021 20:15:15)
3
Initial ONE-MIC auido front-end for speech recognition, mode:1, (Dec 10 2021 20:29:24)
SINGLE_RECOGNITION: v2; (Dec 10 2021 20:45:08)
I (1604) MN: ---------------------SPEECH COMMANDS---------------------
I (1604) MN: Command ID0, phrase 0: da kai kong tiao
I (1614) MN: Command ID1, phrase 1: guan bi kong tiao
I (1614) MN: Command ID2, phrase 2: zeng da feng su
I (1624) MN: Command ID3, phrase 3: jian xiao feng su
I (1634) MN: Command ID4, phrase 4: sheng gao yi du
I (1634) MN: Command ID5, phrase 5: jiang di yi du
I (1654) MN: Command ID6, phrase 6: zhi re mo shi
I (1654) MN: Command ID7, phrase 7: zhi leng mo shi
I (1654) MN: Command ID8, phrase 8: song feng mo shi
I (1654) MN: Command ID9, phrase 9: jie neng mo shi
I (1664) MN: Command ID10, phrase 10: chu shi mo shi
I (1664) MN: Command ID11, phrase 11: jian kang mo shi
I (1674) MN: Command ID12, phrase 12: shui mian mo shi
I (1684) MN: Command ID13, phrase 13: da kai lan ya
I (1684) MN: Command ID14, phrase 14: guan bi lan ya
I (1704) MN: Command ID15, phrase 15: kai shi bo fang
I (1704) MN: Command ID16, phrase 16: zan ting bo fang
I (1704) MN: Command ID17, phrase 17: ding shi yi xiao shi
I (1714) MN: Command ID18, phrase 18: da kai dian deng
I (1714) MN: Command ID19, phrase 19: guan bi dian deng
I (1724) MN: ---------------------------------------------------------

W (1724) MN: MultiNet may cause WatchDog on ESP32 sometimes!
chaitanya-smartrotamac commented 2 years ago

First i got this error Then i enabled multinet from menuconfig

rst:0x1 (POWERON_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:6972
ho 0 tail 12 room 4
load:0x40078000,len:15520
load:0x40080400,len:4492
0x40080400: _init at ??:?

entry 0x400806b4
I (29) boot: ESP-IDF v4.4-329-g3c9657fa4e-dirty 2nd stage bootloader
I (29) boot: compile time 14:54:56
I (29) boot: chip revision: 3
I (34) boot_comm: chip revision: 3, min. bootloader chip revision: 0
I (41) qio_mode: Enabling default flash chip QIO
I (46) boot.esp32: SPI Speed      : 80MHz
I (51) boot.esp32: SPI Mode       : QIO
I (55) boot.esp32: SPI Flash Size : 4MB
I (60) boot: Enabling RNG early entropy source...
I (65) boot: Partition Table:
I (69) boot: ## Label            Usage          Type ST Offset   Length
I (76) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (83) boot:  1 otadata          OTA data         01 00 0000d000 00002000
I (91) boot:  2 phy_init         RF data          01 01 0000f000 00001000
I (98) boot:  3 ota_0            OTA app          00 10 00010000 00300000
I (106) boot:  4 flash_tone       Unknown data     01 27 00310000 00032000
I (113) boot: End of partition table
I (118) boot: No factory image, trying OTA 0
I (123) boot_comm: chip revision: 3, min. application chip revision: 0
I (130) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=3e688h (255624) map
I (209) esp_image: segment 1: paddr=0004e6b0 vaddr=3ffb0000 size=01968h (  6504) load
I (211) esp_image: segment 2: paddr=00050020 vaddr=400d0020 size=74968h (477544) map
I (346) esp_image: segment 3: paddr=000c4990 vaddr=3ffb1968 size=00ec0h (  3776) load
I (348) esp_image: segment 4: paddr=000c5858 vaddr=40080000 size=138f0h ( 80112) load
I (378) esp_image: segment 5: paddr=000d9150 vaddr=50000000 size=00010h (    16) load
I (388) boot: Loaded app from partition at offset 0x10000
I (388) boot: Set actual ota_seq=1 in otadata[0]
I (389) boot: Disabling RNG early entropy source...
I (406) psram: This chip is ESP32-D0WD
I (406) spiram: Found 64MBit SPI RAM device
I (406) spiram: SPI RAM mode: flash 80m sram 80m
I (409) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (416) cpu_start: Pro cpu up.
I (420) cpu_start: Starting app cpu, entry point is 0x400816ec
0x400816ec: call_start_cpu1 at C:/Espressif/frameworks/esp-adf-v2.4/esp-idf/components/esp_system/port/cpu_start.c:160

I (0) cpu_start: App cpu up.
I (902) spiram: SPI SRAM memory test OK
I (909) cpu_start: Pro cpu start user code
I (909) cpu_start: cpu freq: 240000000
I (909) cpu_start: Application information:
I (912) cpu_start: Project name:     example_wwe
I (917) cpu_start: App version:      v2.4-dirty
I (923) cpu_start: Compile time:     May 17 2022 14:54:41
I (929) cpu_start: ELF file SHA256:  471cff04831f51ff...
I (935) cpu_start: ESP-IDF:          v4.4-329-g3c9657fa4e-dirty
I (941) heap_init: Initializing. RAM available for dynamic allocation:
I (948) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (954) heap_init: At 3FFB4880 len 0002B780 (173 KiB): DRAM
I (961) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (967) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (973) heap_init: At 400938F0 len 0000C710 (49 KiB): IRAM
I (980) spiram: Adding pool of 4095K of external SPI memory to heap allocator
I (988) spi_flash: detected chip: generic
I (992) spi_flash: flash io: qio
W (996) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
I (1010) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (1020) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1030) gpio: GPIO[36]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1040) gpio: GPIO[39]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1050) gpio: GPIO[19]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
E (1070) gpio: gpio_install_isr_service(449): GPIO isr service already installed
W (1070) PERIPH_TOUCH: _touch_init
I (1080) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (1080) ES8388_DRIVER: init,out:02, in:00
W (1090) AUDIO_BOARD: The board has already been initialized!

----------------------------- ESP Audio Platform -----------------------------
|                                                                            |
|                 ESP_AUDIO-v1.7.0-63-g2b9ee8e-bec4a56-4d51bc8               |
|                     Compile date: Feb 23 2022-02:43:28                     |
------------------------------------------------------------------------------
I (1130) LYRAT_V4_3: I2S0, MCLK output by GPIO0
I (1140) wwe_example: Func:setup_player, Line:131, MEM Total:4426539 Bytes, Inter:316979 Bytes, Dram:268171 Bytes

I (1150) wwe_example: esp_audio instance is:0x3f801b44

E (1150) I2S: register I2S object to platform failed
I (1170) LYRAT_V4_3: I2S0, MCLK output by GPIO0
I (1170) wwe_example: Recorder has been created
Deprecated Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0 (Dec 10 2021 20:15:15)
3
Initial ONE-MIC auido front-end for speech recognition, mode:1, (Dec 10 2021 20:29:24)
W (1290) RECORDER_SR: Multinet is not enabled in SDKCONFIG
HengYongChao commented 2 years ago

Hi @chaitanya-smartrotamac

First, update your esp-sr submodule by using command: git submodule update --init --recursive.

Then, setup menuconfig like below, there is a same question FYI #814 .

image

chaitanya-smartrotamac commented 2 years ago

Hi @HengYongChao Thank you for the response. The example worked. The indication should be added like after W (1724) MN: MultiNet may cause WatchDog on ESP32 sometimes!

After this say ' Hi Lexin ' I missed that. I thought I should say that chinese phrases were given.

Now is there a way we can customize the wake word '' Hi Lexin ''?

HengYongChao commented 2 years ago

Hi @HengYongChao Thank you for the response. The example worked. The indication should be added like after W (1724) MN: MultiNet may cause WatchDog on ESP32 sometimes!

After this say ' Hi Lexin ' I missed that. I thought I should say that chinese phrases were given.

Now is there a way we can customize the wake word '' Hi Lexin ''?

HI @chaitanya-smartrotamac

Please see here. https://github.com/espressif/esp-sr/blob/master/README.md

Wake Word Engine

Espressif wake word engine WakeNet is specially designed to provide a high performance and low memory footprint wake word detection algorithm for users, which enables devices always listen wake words, such as “Alexa”, “天猫精灵” (Tian Mao Jing Ling) and “小爱同学” (Xiao Ai Tong Xue). You can refer to Model loading method to build your project.

Currently, Espressif has not only provided an official wake word "Hi,Lexin","Hi,ESP" to public for free, but also allows customized wake words. For details on how to customize your own wake words, please see Espressif Speech Wake Words Customization Process.