ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad (AUD-5627)

HyouinSchoolAcc commented 3 weeks ago

Environment

Audio development kit: ESP32-LyraT-Mini-1.2
[Required] Module or chip used: ESP32-WROVER-E
[Required] IDF version v5.2.2
[Required] ADF version v2.6-159-gdac74d81
Build system: [Make|CMake|idf.py]
[Required] Running log:
Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it): // 1.22.0-80-g6c4433a
Operating system: [Windows|Linux|macOS]
(Windows only) Environment type: [MSYS2 mingw32|ESP Command Prompt|Plain Command Prompt|PowerShell]
Using an IDE?: [No|Yes (please give details)]
Power supply: [USB|external 5V|external 3.3V|Battery]

Problem Description

When using two examples that utilizes the microphone, I've noticed that all the pipelines are working but the microphone itself is not providing any sound. I bought a second lyrat mini 1.2 to confirm this isnt a unique hardware issue. It seems that the i2s interface is working fine given that the speaker can be played.

Expected Behavior

When the board is programmed the microphone is supposed to pick up sound

Actual Behavior

All sound recorded/inputted are null and the sound audio is completely silent

Steps to Reproduce

get an esp32-lyrat-mini-v1.2
upload raw_http example or the voice_recognition/vad example
in the raw_http example the recorded voice will be blank. In the vad example there will not be any updates after "vad setup and running"

// If possible, attach a picture of your setup/wiring here.

Code to Reproduce This Issue

All code are exactly as written as the example given in examples raw_http and vad.

Debug Logs

I (10) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader I (10) boot: compile time Aug 16 2024 ï¿½I (10) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader I (10) boot: compile time Aug 16 2024 11:49:03 I (10) boot: Multicore bootloader I (14) boot: chip revision: v3.1 I (18) boot.esp32: SPI Speed : 80MHz I (23) boot.esp32: SPI Mode : DIO I (27) boot.esp32: SPI Flash Size : 8MB I (32) boot: Enabling RNG early entropy source... I (37) boot: Partition Table: I (41) boot: ## Label Usage Type ST Offset Length I (48) boot: 0 nvs WiFi data 01 02 00009000 00006000 I (55) boot: 1 phy_init RF data 01 01 0000f000 00001000 I (63) boot: 2 factory factory app 00 00 00010000 00200000 I (70) boot: End of partition table I (74) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=2bff0h (180208) map I (109) esp_image: segment 1: paddr=0003c018 vaddr=3ffb0000 size=03f98h ( 16280) load I (112) esp_image: segment 2: paddr=0003ffb8 vaddr=40080000 size=00060h ( 96) load I (115) esp_image: segment 3: paddr=00040020 vaddr=400d0020 size=a7314h (684820) map I (224) esp_image: segment 4: paddr=000e733c vaddr=40080060 size=1a0cch (106700) load I (258) boot: Loaded app from partition at offset 0x10000 I (258) boot: Disabling RNG early entropy source... I (269) cpu_start: Multicore app I (269) quad_psram: This chip is ESP32-D0WD I (270) esp_psram: Found 8MB PSRAM device I (271) esp_psram: Speed: 40MHz I (275) esp_psram: PSRAM initialized, cache is in low/high (2-core) mode. W (283) esp_psram: Virtual address not enough for PSRAM, map as much as we can. 4MB is mapped I (1172) esp_psram: SPI SRAM memory test OK I (1179) cpu_start: Pro cpu start user code I (1179) cpu_start: cpu freq: 160000000 Hz I (1180) cpu_start: Application information: I (1183) cpu_start: Project name: record_raw_http I (1188) cpu_start: App version: 1 I (1193) cpu_start: Compile time: Aug 19 2024 21:34:02 I (1199) cpu_start: ELF file SHA256: c6650c452... I (1205) cpu_start: ESP-IDF: v5.2.2-dirty I (1210) cpu_start: Min chip rev: v0.0 I (1215) cpu_start: Max chip rev: v3.99 I (1220) cpu_start: Chip rev: v3.1 I (1225) heap_init: Initializing. RAM available for dynamic allocation: I (1232) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM I (1238) heap_init: At 3FFB8A00 len 00027600 (157 KiB): DRAM I (1244) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM I (1251) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM I (1257) heap_init: At 4009A12C len 00005ED4 (23 KiB): IRAM I (1264) esp_psram: Adding pool of 4096K of PSRAM memory to heap allocator I (1272) spi_flash: detected chip: gd I (1275) spi_flash: flash io: dio W (1280) i2c: This driver is an old driver, please migrate your application code to adapt driver/i2c_master.h W (1290) ADC: legacy driver is deprecated, please migrate to esp_adc/adc_oneshot.h I (1299) main_task: Started on CPU0 I (1309) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations I (1309) main_task: Calling app_main() I (1359) REC_RAW_HTTP: [ 1 ] Initialize Button Peripheral & Connect to wifi network W (1459) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41 W (2679) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41 W (2829) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:4 I (4329) REC_RAW_HTTP: [ 2 ] Start codec chip W (4349) I2C_BUS: I2C bus has been already created, [port:0] I (4369) REC_RAW_HTTP: [3.0] Create audio pipeline for recording I (4369) REC_RAW_HTTP: [3.1] Create http stream to post data to server I (4369) REC_RAW_HTTP: [3.2] Create i2s stream to read audio data from codec chip I (4379) REC_RAW_HTTP: [3.3] Register all elements to audio pipeline I (4379) REC_RAW_HTTP: [3.4] Link it together [codec_chip]-->i2s_stream->http_stream-->[http_server] I (4399) REC_RAW_HTTP: Volume set successfully to 100. I (4399) REC_RAW_HTTP: Current volume is: 100 I (4399) REC_RAW_HTTP: [ 4 ] Press [Rec] button to record, Press [Mode] to exit W (17339) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41 E (53339) REC_RAW_HTTP: [ * ] [Rec] input key event, resuming pipeline ... W (53339) AUDIO_PIPELINE: Without stop, st:1 W (53339) AUDIO_PIPELINE: Without wait stop, st:1 W (53339) AUDIO_ELEMENT: [i2s] Element has not create when AUDIO_ELEMENT_TERMINATE W (53349) AUDIO_ELEMENT: [http] Element has not create when AUDIO_ELEMENT_TERMINATE W (53359) AUDIO_THREAD: Make sure selected the CONFIG_SPIRAM_BOOT_INIT and CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY by make menuconfig Total bytes written: 61440 Total bytes written: 64800 I (55429) REC_RAW_HTTP: [ + ] HTTP client HTTP_STREAM_POST_REQUEST, write end chunked marker I (55439) REC_RAW_HTTP: [ + ] HTTP client HTTP_STREAM_FINISH_REQUEST W (55449) HTTP_CLIENT: esp_transport_read returned:-1 and errno:128 I (55449) REC_RAW_HTTP: Got HTTP Response = File 20240820T032231Z_16000_16_1.wav was written, size 64800

Other Items If Possible

[x] sdkconfig file (Attach the sdkconfig file from your project folder)
[x] elf file in the build folder (Note this may contain all the code details and symbols of your project.)
[x] coredump (This provides stacks of tasks.)

HyouinSchoolAcc commented 3 weeks ago

sdkconfig.json record_raw_http.zip no crashes - coredump not included

jason-mao commented 3 weeks ago

@HyouinSchoolAcc Would please test example from latest master branch?

HyouinSchoolAcc commented 3 weeks ago

I have tested the latest master branch esp-adf. My steps:

git pull from adf
delete current workspace copy
import project, copy into workspace
cd into workspace in terminal, idf.py menuconfig -> modify the server address, ssid and password, and config board to lyrat-mini 1.1 board.
flash onto target esp32

The audio returned from the webserver is still silent

HyouinSchoolAcc commented 3 weeks ago

I'm looking at the diagram and it appears that the I2S peripheral is connected to the ES7243 ADC, and I think it might be helpful to probe the ADC to make sure its responding correctly. I added logI''s to i2s_stream.c under i2s_read but it appears it is not being registered as my logs never show up during recording.

HyouinSchoolAcc commented 3 weeks ago

Upon further investigation, it appears that this board, the v1.2, needs its ES7243 initialized with I2C register writes in order to receive audio. I read some documentations in ES7243 and is now able to turn on the ADC chip. I still don't know how it interfaces with the codec chip though.

Bhavesh-Moradiya commented 3 weeks ago

I am working on same board and having issue with i2s, I want raw data but because of 2 different IC used in no luck

Bhavesh-Moradiya commented 3 weeks ago

I checked BSP packages and found ES8311 I2C address is 0x30 and es7243 0x26 but according to schematic it should be 18 and 20 respectively, if any idea about this please let me know what I am missing

HyouinSchoolAcc commented 2 weeks ago

I really couldn't figure out how to initiate the codec chip after 3 days of trying, how about you? Were you able to recieve any audio? I havn't tried to force the pipeline to use the ES7243 chip, but that theoretically is easily achievable. How about you? Any plans?

From: Bhavesh M Moradiya @.> Sent: Saturday, August 24, 2024 11:45 AM To: espressif/esp-adf @.> Cc: Liu, Hyouin @.>; Mention @.> Subject: Re: [espressif/esp-adf] ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad (AUD-5627) (Issue #1252)

I checked BSP packages and found ES8311 I2C address is 0x30 and es7243 0x26 but according to schematic it should be 18 and 20 respectively, if any idea about this please let me know what I am missing

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/espressif/esp-adf/issues/1252*issuecomment-2308452601__;Iw!!DZ3fjg!4UrPTyaC-PV5rDDFEDasT_abA95BjsY0yYGUuzYlrotmitxNqTiFqGP7tjJyIZ5kEO7OLUoUpo7DmFwxXqSt2pG6VNBs$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BAOY3LGTAL5HSNF23LEETILZTC2DTAVCNFSM6AAAAABMY73PN2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBYGQ2TENRQGE__;!!DZ3fjg!4UrPTyaC-PV5rDDFEDasT_abA95BjsY0yYGUuzYlrotmitxNqTiFqGP7tjJyIZ5kEO7OLUoUpo7DmFwxXqSt2i2w3V-Y$. You are receiving this because you were mentioned.Message ID: @.***>

espressif / esp-adf