WakeNet 8-bit Quantized: "Item psram alloc failed" on ESP32-S3-Wroom-1-N8R2 (AIS-864)

[ghost]

I am using a custom board with 2 I2S microphones. Havn't set up microphones yet, compiling the WakeWord example with ESP32-S3-EYE set as board in menuconfig. I have Disable MultiNet since I have only 2MB of RAM available, but this should be ample for WakeNet, correct? Then I don't underestand why I get "Item psram alloc failed" from model. Am I running out of PSRAM?? I adjusted many settings in menuconfig, but still get same error.

ELF file SHA256: 90725b566f0ff608

Rebooting... xESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT) Saved PC:0x403799e0 SPIWP:0xee mode:DIO, clock div:1 load:0x3fcd0108,len:0x17b4 load:0x403b6000,len:0xe28 load:0x403ba000,len:0x30d0 entry 0x403b6274 I (29) boot: ESP-IDF v4.4-348-gc45dee0a7a 2nd stage bootloader I (29) boot: chip revision: 0 I (32) qio_mode: Enabling default flash chip QIO I (37) boot.esp32s3: Boot SPI Speed : 80MHz I (42) boot.esp32s3: SPI Mode : QIO I (47) boot.esp32s3: SPI Flash Size : 8MB I (52) boot: Enabling RNG early entropy source... I (57) boot: Partition Table: I (61) boot: ## Label Usage Type ST Offset Length I (68) boot: 0 factory factory app 00 00 00010000 00271000 I (75) boot: 1 model Unknown data 01 82 00281000 0050c000 I (83) boot: End of partition table I (87) esp_image: segment 0: paddr=00010020 vaddr=3c060020 size=1f420h (128032) map I (115) esp_image: segment 1: paddr=0002f448 vaddr=3fc9a800 size=00bd0h ( 3024) load I (116) esp_image: segment 2: paddr=00030020 vaddr=42000020 size=5e6a8h (386728) map I (179) esp_image: segment 3: paddr=0008e6d0 vaddr=3fc9b3d0 size=01c4ch ( 7244) load I (181) esp_image: segment 4: paddr=00090324 vaddr=40378000 size=127f4h ( 75764) load I (199) esp_image: segment 5: paddr=000a2b20 vaddr=50000000 size=00010h ( 16) load I (207) boot: Loaded app from partition at offset 0x10000 I (207) boot: Disabling RNG early entropy source... I (220) spiram: Found 16MBit SPI RAM device I (220) spiram: SPI RAM mode: sram 80m I (220) spiram: PSRAM initialized, cache is in normal (1-core) mode. I (225) cpu_start: Pro cpu up. I (228) cpu_start: Starting app cpu, entry point is 0x4037932c I (213) cpu_start: App cpu up. I (464) spiram: SPI SRAM memory test OK I (465) spiram: Instructions copied and mapped to SPIRAM I (510) spiram: Read only data copied and mapped to SPIRAM I (525) cache: SPIRAM wrap enabled, size = 32. I (534) cpu_start: Pro cpu start user code I (534) cpu_start: cpu freq: 160000000 I (534) cpu_start: Application information: I (537) cpu_start: Project name: wake_word_detection I (543) cpu_start: App version: v0.3.0-105-g4847f52-dirty I (555) cpu_start: ELF file SHA256: 90725b566f0ff608... I (561) cpu_start: ESP-IDF: v4.4-348-gc45dee0a7a I (568) heap_init: Initializing. RAM available for dynamic allocation: I (575) heap_init: At 3FC9F058 len 00040FA8 (259 KiB): D/IRAM I (581) heap_init: At 3FCE0000 len 0000EE34 (59 KiB): STACK/DRAM I (588) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM I (594) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM I (600) spiram: Adding pool of 1536K of external SPI memory to heap allocator I (608) spi_flash: detected chip: gd I (612) spi_flash: flash io: qio I (617) sleep: Configure to isolate all GPIO pins in sleep state I (623) sleep: Enable automatic switching of GPIO sleep configuration I (630) cpu_start: Starting scheduler on PRO CPU. I (0) cpu_start: Starting scheduler on APP CPU. I (641) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations Initializing SPIFFS Partition size: total: 4857101, used: 874233 I (1121) I2S: DMA Malloc info, datalen=blocksize=640, dma_buf_count=6 model_name: hiesp8q8 model_data: /srmodel/hiesp8q8/wn8q8_data Item psram alloc failed. Size: 163860 = 81920 x 2 + 16 + 4 item buff malloc fail Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled.

Core 0 register dump: PC : 0x40056fcc PS : 0x00060f30 A0 : 0x82008934 A1 : 0x3fcf3f10
A2 : 0x00000000 A3 : 0x3dede104 A4 : 0x00028000 A5 : 0x00000000
A6 : 0x0101fb0f A7 : 0x0a000101 A8 : 0xf90a060a A9 : 0x3fcf3ec0
A10 : 0x0000000a A11 : 0x00000002 A12 : 0x00014000 A13 : 0x00000002
A14 : 0x00000010 A15 : 0x00000004 SAR : 0x00000010 EXCCAUSE: 0x0000001d
EXCVADDR: 0x00000000 LBEG : 0x40056fc5 LEND : 0x40056fe7 LCOUNT : 0x000027ff

Backtrace:0x40056fc9:0x3fcf3f10 |<-CORRUPTED

[LiTongXue98]

menuconfig 里面的 CONFIG_ESP32S3_SPIRAM_SUPPORT=y 有没有配置呢？

[ghost]

Yes, I have already done that:

% % ESP32S3-Specific % % CONFIG_ESP32S3_DEFAULT_CPU_FREQ_80 is not set CONFIG_ESP32S3_DEFAULT_CPU_FREQ_160=y % CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240 is not set CONFIG_ESP32S3_DEFAULT_CPU_FREQ_MHZ=160

% % Cache config % % CONFIG_ESP32S3_INSTRUCTION_CACHE_16KB is not set CONFIG_ESP32S3_INSTRUCTION_CACHE_32KB=y CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE=0x8000 % CONFIG_ESP32S3_INSTRUCTION_CACHE_4WAYS is not set CONFIG_ESP32S3_INSTRUCTION_CACHE_8WAYS=y CONFIG_ESP32S3_ICACHE_ASSOCIATED_WAYS=8 CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_32B=y CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_SIZE=32 CONFIG_ESP32S3_INSTRUCTION_CACHE_WRAP=y % CONFIG_ESP32S3_DATA_CACHE_16KB is not set CONFIG_ESP32S3_DATA_CACHE_32KB=y % CONFIG_ESP32S3_DATA_CACHE_64KB is not set CONFIG_ESP32S3_DATA_CACHE_SIZE=0x8000 % CONFIG_ESP32S3_DATA_CACHE_4WAYS is not set CONFIG_ESP32S3_DATA_CACHE_8WAYS=y CONFIG_ESP32S3_DCACHE_ASSOCIATED_WAYS=8 % CONFIG_ESP32S3_DATA_CACHE_LINE_16B is not set CONFIG_ESP32S3_DATA_CACHE_LINE_32B=y % CONFIG_ESP32S3_DATA_CACHE_LINE_64B is not set CONFIG_ESP32S3_DATA_CACHE_LINE_SIZE=32 CONFIG_ESP32S3_DATA_CACHE_WRAP=y % end of Cache config

CONFIG_ESP32S3_SPIRAM_SUPPORT=y

% % SPI RAM config % CONFIG_SPIRAM_MODE_QUAD=y % CONFIG_SPIRAM_MODE_OCT is not set CONFIG_SPIRAM_TYPE_AUTO=y % CONFIG_SPIRAM_TYPE_ESPPSRAM16 is not set % CONFIG_SPIRAM_TYPE_ESPPSRAM32 is not set % CONFIG_SPIRAM_TYPE_ESPPSRAM64 is not set CONFIG_SPIRAM_SIZE=-1

% % PSRAM Clock and CS IO for ESP32S3 % CONFIG_DEFAULT_PSRAM_CLK_IO=30 CONFIG_DEFAULT_PSRAM_CS_IO=26 % end of PSRAM Clock and CS IO for ESP32S3

CONFIG_SPIRAM_FETCH_INSTRUCTIONS=y CONFIG_SPIRAM_RODATA=y % CONFIG_SPIRAM_SPEED_120M is not set CONFIG_SPIRAM_SPEED_80M=y % CONFIG_SPIRAM_SPEED_40M is not set CONFIG_SPIRAM=y CONFIG_SPIRAM_BOOT_INIT=y % CONFIG_SPIRAM_IGNORE_NOTFOUND is not set % CONFIG_SPIRAM_USE_MEMMAP is not set % CONFIG_SPIRAM_USE_CAPS_ALLOC is not set CONFIG_SPIRAM_USE_MALLOC=y CONFIG_SPIRAM_MEMTEST=y CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=131072 CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768 % end of SPI RAM config

% CONFIG_ESP32S3_TRAX is not set CONFIG_ESP32S3_TRACEMEM_RESERVE_DRAM=0x0 % CONFIG_ESP32S3_ULP_COPROC_ENABLED is not set CONFIG_ESP32S3_ULP_COPROC_RESERVE_MEM=0 CONFIG_ESP32S3_DEBUG_OCDAWARE=y CONFIG_ESP32S3_BROWNOUT_DET=y CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_7=y % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_6 is not set % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_5 is not set % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_4 is not set % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_3 is not set % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_2 is not set % CONFIG_ESP32S3_BROWNOUT_DET_LVL_SEL_1 is not set CONFIG_ESP32S3_BROWNOUT_DET_LVL=7 CONFIG_ESP32S3_TIME_SYSCALL_USE_RTC_FRC1=y % CONFIG_ESP32S3_TIME_SYSCALL_USE_RTC is not set % CONFIG_ESP32S3_TIME_SYSCALL_USE_FRC1 is not set % CONFIG_ESP32S3_TIME_SYSCALL_USE_NONE is not set CONFIG_ESP32S3_RTC_CLK_SRC_INT_RC=y % CONFIG_ESP32S3_RTC_CLK_SRC_EXT_CRYS is not set % CONFIG_ESP32S3_RTC_CLK_SRC_EXT_OSC is not set % CONFIG_ESP32S3_RTC_CLK_SRC_INT_8MD256 is not set CONFIG_ESP32S3_RTC_CLK_CAL_CYCLES=1024 CONFIG_ESP32S3_DEEP_SLEEP_WAKEUP_DELAY=2000 % CONFIG_ESP32S3_NO_BLOBS is not set % CONFIG_ESP32S3_RTCDATA_IN_FAST_MEM is not set % CONFIG_ESP32S3_USE_FIXED_STATIC_RAM_SIZE is not set % end of ESP32S3-Specific

[feizi]

8bit-wakenet8 takes about 850KB, Audio-Front-End module takes about 500 KB. We haven't tested the R8N2, but it's possible to run out of memory.

[feizi]

Next month, we will release a wakenet9 in v2.0 which takes about 300 KB. If you don't want to change ESP32-S3 module, you can wait for the latest wakenet9.

[ghost]

Thanks, I wish I knew about this, now I am left with custom boards that are no good 😔, I made them specifically to use a single wake word (i.e. Alexa) I read in https://www.espressif.com/en/solutions/audio-solutions/esp-skainet/overview and https://www.cnx-software.com/2019/09/02/espressif-esp-skainet-voice-assistant-wake-word-engine-speech-commands-recognition-embedded-mcus/

that RAM usage for WakeWord is <100KB, hence choosing N8R2. Anyways, please support "Alexa" and "Hi, ESP" in WakeNet9. Good to know that there is still some hope.

[feizi]

Hi eyewy， I think I need to explain the details about memory for different wakenet. wakenet5(ESP32): we read all parameters from flash and do not save a backup in PSRAM, so we only need RAM < 100KB. wakenet7&wakenet8(ESP32S3), When we initialize the model, parameters are read into PSRAM from flash(the parameter of wakenet8 is about 800KB), then we use the parameters in PSRAM to calculate.

Why do we do that on ESP32-S3? The main reason is that Octal PSRAM is faster than QIO flash. And Octal flash is too expensive.

[ghost]

Tested on ESP32-S3-DevKitC-1-N32R8V Development Board, and there is no RAM issue any more. This board says: I (729) spiram: Adding pool of 8192K of external SPI memory to heap allocator After running the program I check RAM usage by: printf("RAM left %d\n", esp_get_free_heap_size()); and I get: RAM left 6718923 So actual ram usage is: 1473KB

This confirms @feizi calculation of 850KB + 500KB =1350KB

Now here is my question: given ~1.5MB usage which is <2MB, how come I receive error on N8R2?? Is there a temperorary peak usage while loading the model? Is there something that can be done here to optimize instantaneous RAM usage?

Also, I don't use Wifi and BT, is there a way to free up little bit more memory?

If Answer to above questions is no, I guess I can only wait for WakeNet9 to become available, I would be happy to test it on N8R2 as soon as you have a beta version available.

[feizi]

Yes, when loading the model, it does require more memory. I am sorry to tell you wakenet9 will be delayed. Colleagues in Shanghai can not upload new codes due to COVID-19.

[feizi]

Hi @eyewy ， wakenet9 has been updated in the master branch.