search

espressif / esp-adf

Espressif Audio Development Framework
Other
1.49k stars 667 forks source link

wwe 录音保存到pcm有断续丢帧的问题, pipeline_recording_to_sdcard 也有类似的问题, advanced_examples\algorithm 正常录音 #1156

Closed Henry586 closed 4 months ago

Henry586 commented 4 months ago

基于最新的adf版本 2024/02/02, 板子为 ESP32-S3-KORVO-2_V3.0

修改wwe使能VOICE2FILE,禁止了MULTINET.

define VOICE2FILE (true)

define WAKENET_ENABLE (true)

define MULTINET_ENABLE (false)

define SPEECH_CMDS_RESET (false)

日志并没有异常: I (1776) AFE_SR: wake num: 3, mode: 0, (Jan 12 2024 18:00:46)

I (1796) main_task: Returned from app_main() I (8586) wwe_example: rec_engine_cb - REC_EVENT_WAKEUP_START I (8596) wwe_example: wakeup: vol -18.701904, mod idx 1, word idx 1 I (8606) CODEC_ELEMENT_HELPER: The element is 0x3c0e9abc. The reserve data 2 is 0x0. I (11086) wwe_example: rec_engine_cb - REC_EVENT_VAD_START W (11096) wwe_example: voice read begin I (15956) wwe_example: rec_engine_cb - REC_EVENT_VAD_STOP W (15966) wwe_example: voice read stopped I (15966) wwe_example: File closed I (16856) wwe_example: rec_engine_cb - REC_EVENT_WAKEUP_END I (16856) wwe_example: Func:rec_engine_cb, Line:266, MEM Total:7353424 Bytes, Inter:244739 Bytes, Dram:244739 Bytes

但是录音得到的 F0.PCM 感觉有断续，用audacity以16kbps,16bit导入，听起来有断续丢帧，尤其是录音中后面的几秒钟。

请问这个是否和board_def.h中的采样率和采样bit的设置有关，如果降低采样率为16k,采样bit为16bit的话，main.c还需要做什么样的调整呢?

define CODEC_ADC_BITS_PER_SAMPLE (32) / 32bit /

define CODEC_ADC_SAMPLE_RATE (48000)

谢谢！

Henry586 commented 4 months ago

例子 esp-adf\examples\recorder\pipeline_recording_to_sdcard 只是修改输出为wav,没有修改其他的配置，录音得到的wav文件也有断续丢帧的问题。 日志:

`ESP-ROM:esp32s3-20210327 Build:Mar 27 2021 rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT) SPIWP:0xee mode:DIO, clock div:1 load:0x3fce3818,len:0x18e4 load:0x403c9700,len:0x4 load:0x403c9704,len:0xda4 load:0x403cc700,len:0x3084 entry 0x403c9934 I (27) boot: ESP-IDF v5.1.2-dirty 2nd stage bootloader I (27) boot: compile time Feb 26 2024 21:18:12 I (27) boot: Multicore bootloader I (30) boot: chip revision: v0.1 I (34) qio_mode: Enabling default flash chip QIO I (39) boot.esp32s3: Boot SPI Speed : 80MHz I (44) boot.esp32s3: SPI Mode : QIO I (49) boot.esp32s3: SPI Flash Size : 4MB I (54) boot: Enabling RNG early entropy source... I (59) boot: Partition Table: I (63) boot: ## Label Usage Type ST Offset Length I (70) boot: 0 nvs WiFi data 01 02 00009000 00006000 I (77) boot: 1 phy_init RF data 01 01 0000f000 00001000 I (85) boot: 2 factory factory app 00 00 00010000 00100000 I (92) boot: End of partition table I (97) esp_image: segment 0: paddr=00010020 vaddr=3c040020 size=409e0h (264672) map I (145) esp_image: segment 1: paddr=00050a08 vaddr=3fc96700 size=02f44h ( 12100) load I (148) esp_image: segment 2: paddr=00053954 vaddr=40374000 size=0c6c4h ( 50884) load I (161) esp_image: segment 3: paddr=00060020 vaddr=42000020 size=3e634h (255540) map I (200) esp_image: segment 4: paddr=0009e65c vaddr=403806c4 size=05fa4h ( 24484) load I (212) boot: Loaded app from partition at offset 0x10000 I (213) boot: Disabling RNG early entropy source... I (224) cpu_start: Multicore app I (224) octal_psram: vendor id : 0x0d (AP) I (225) octal_psram: dev id : 0x02 (generation 3) I (228) octal_psram: density : 0x03 (64 Mbit) I (233) octal_psram: good-die : 0x01 (Pass) I (238) octal_psram: Latency : 0x01 (Fixed) I (244) octal_psram: VCC : 0x01 (3V) I (249) octal_psram: SRF : 0x01 (Fast Refresh) I (255) octal_psram: BurstType : 0x01 (Hybrid Wrap) I (260) octal_psram: BurstLen : 0x01 (32 Byte) I (266) octal_psram: Readlatency : 0x02 (10 cycles@Fixed) I (272) octal_psram: DriveStrength: 0x00 (1/1) I (278) MSPI Timing: PSRAM timing tuning index: 4 I (283) esp_psram: Found 8MB PSRAM device I (287) esp_psram: Speed: 80MHz I (291) cpu_start: Pro cpu up. I (295) cpu_start: Starting app cpu, entry point is 0x403756a4 I (0) cpu_start: App cpu up. 0x403756a4: call_start_cpu1 at C:/Espressif/frameworks/esp-idf-v5.1.2/components/esp_system/port/cpu_start.c:157

I (747) esp_psram: SPI SRAM memory test OK I (756) cpu_start: Pro cpu start user code I (756) cpu_start: cpu freq: 160000000 Hz I (757) cpu_start: Application information: I (759) cpu_start: Project name: recording_to_sdcard I (765) cpu_start: App version: v2.6-82-g33ba0c5c-dirty I (772) cpu_start: Compile time: Feb 26 2024 21:17:32 I (778) cpu_start: ELF file SHA256: 1cc807afdd11bcbb... I (784) cpu_start: ESP-IDF: v5.1.2-dirty I (789) cpu_start: Min chip rev: v0.0 I (794) cpu_start: Max chip rev: v0.99 I (799) cpu_start: Chip rev: v0.1 I (804) heap_init: Initializing. RAM available for dynamic allocation: I (811) heap_init: At 3FC9A360 len 0004F3B0 (316 KiB): DRAM I (817) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM I (824) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM I (830) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM I (836) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator I (844) spi_flash: detected chip: gd I (848) spi_flash: flash io: qio W (852) spi_flash: Detected size(16384k) larger than the size in the binary image header(4096k). Using the size in the binary image header. W (865) ADC: legacy driver is deprecated, please migrate to esp_adc/adc_oneshot.h I (873) sleep: Configure to isolate all GPIO pins in sleep state I (880) sleep: Enable automatic switching of GPIO sleep configuration I (887) app_start: Starting scheduler on CPU0 I (892) app_start: Starting scheduler on CPU1 I (892) main_task: Started on CPU0 I (902) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations I (910) main_task: Calling app_main() I (914) RECORD_TO_SDCARD: [ 1 ] Mount sdcard I (1420) RECORD_TO_SDCARD: [ 2 ] Start codec chip W (1443) I2C_BUS: I2C bus has been already created, [port:0] W (1469) ES7210: Enable TDM mode. ES7210_SDP_INTERFACE2_REG12: 2 I (1487) RECORD_TO_SDCARD: [3.0] Create audio pipeline for recording I (1487) RECORD_TO_SDCARD: [3.1] Create fatfs stream to write data to sdcard I (1491) RECORD_TO_SDCARD: [3.2] Create i2s stream to read audio data from codec chip I (1500) RECORD_TO_SDCARD: [3.3] Create audio encoder to handle data I (1507) RECORD_TO_SDCARD: [3.4] Register all elements to audio pipeline I (1514) RECORD_TO_SDCARD: [3.5] Link it together [codec_chip]-->i2s_stream-->audio_encoder-->fatfs_stream-->[sdcard] I (1526) RECORD_TO_SDCARD: [3.6] Set music info to fatfs I (1531) RECORD_TO_SDCARD: [ ] Save the recording info to the fatfs stream writer, sample_rates=16000, bits=16, ch=2 I (1542) RECORD_TO_SDCARD: [3.7] Set up uri I (1547) RECORD_TO_SDCARD: [ 4 ] Set up event listener I (1553) RECORD_TO_SDCARD: [4.1] Listening event from pipeline I (1560) RECORD_TO_SDCARD: [4.2] Listening event from peripherals I (1567) RECORD_TO_SDCARD: [ 5 ] Start audio_pipeline I (1574) RECORD_TO_SDCARD: [ 6 ] Listen for all pipeline events, record for 10 Seconds I (2850) RECORD_TO_SDCARD: [ ] Recording ... 1 I (4050) RECORD_TO_SDCARD: [ ] Recording ... 2 I (5050) RECORD_TO_SDCARD: [ ] Recording ... 3 I (6080) RECORD_TO_SDCARD: [ ] Recording ... 4 I (7184) RECORD_TO_SDCARD: [ ] Recording ... 5 I (8284) RECORD_TO_SDCARD: [ ] Recording ... 6 I (9314) RECORD_TO_SDCARD: [ ] Recording ... 7 I (10507) RECORD_TO_SDCARD: [ ] Recording ... 8 I (11556) RECORD_TO_SDCARD: [ ] Recording ... 9 I (12651) RECORD_TO_SDCARD: [ ] Recording ... 10 W (13983) RECORD_TO_SDCARD: [ ] Stop event received I (13983) RECORD_TO_SDCARD: [ 7 ] Stop audio_pipeline W (13983) AUDIO_ELEMENT: [i2s] Element already stopped W (13989) AUDIO_ELEMENT: [wav] Element already stopped W (13995) AUDIO_ELEMENT: [file] Element already stopped W (14001) AUDIO_PIPELINE: There are no listener registered W (14007) AUDIO_ELEMENT: [file] Element has not create when AUDIO_ELEMENT_TERMINATE W (14015) AUDIO_ELEMENT: [i2s] Element has not create when AUDIO_ELEMENT_TERMINATE W (14024) AUDIO_ELEMENT: [wav] Element has not create when AUDIO_ELEMENT_TERMINATE None Waiting for the device to reconnect................................................................................ `

Henry586 commented 4 months ago

尝试了一下 esp-adf\examples\advanced_examples\algorithm, 它是可以正常录音的，花点时间比较一下设置上有什么差异

Henry586 commented 4 months ago

按照FAE的提示，这个断续可能和tf卡的写入速度有关， advanced_examples\algorithm 的wave速率是128kbps pipeline_recording_to_sdcard 的wave速率是512kbps wwe 的wave 的速率是 256kbbs 使用的tf卡是 kingston class 4 已经订购了高速卡试试看 参考: https://esp32.com/viewtopic.php?t=26351

Henry586 commented 4 months ago

新的tf卡到了，规格 class 10，录音正常，没有问题了。

Henry586 commented 4 months ago

用例子sdmmc测速 https://github.com/espressif/esp-idf/blob/master/examples/storage/sd_card/sdmmc/

class 4的写入速率不稳定，最低还看到过0.36MB/s: ` Name: SA08G Type: SDHC/SDXC Speed: 40.00 MHz (limit: 40.00 MHz) Size: 7592MB CSD: ver=2, sector_size=512, capacity=15548416 read_bl_len=9 SSR: bus_width=1 I (815) example: Opening file /sdcard/hello.txt I (1885) example: File written speed: 0.95 MB/s I (2615) example: Renaming file /sdcard/hello.txt to /sdcard/foo.txt I (2625) example: Reading file /sdcard/foo.txt I (2625) example: Read from file: 'BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB' I (2625) example: Card unmounted I (2635) main_task: Returned from app_main()

Name: SA08G Type: SDHC/SDXC Speed: 40.00 MHz (limit: 40.00 MHz) Size: 7592MB CSD: ver=2, sector_size=512, capacity=15548416 read_bl_len=9 SSR: bus_width=1 I (818) example: Opening file /sdcard/hello.txt I (1218) example: File written speed: 2.57 MB/s I (4018) example: Renaming file /sdcard/hello.txt to /sdcard/foo.txt I (4028) example: Reading file /sdcard/foo.txt I (4028) example: Read from file: 'BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB' I (4028) example: Card unmounted I (4038) main_task: Returned from app_main() `

class10的写入速度稳定: ` Name: SK32G Type: SDHC/SDXC Speed: 40.00 MHz (limit: 40.00 MHz) Size: 30436MB CSD: ver=2, sector_size=512, capacity=62333952 read_bl_len=9 SSR: bus_width=1 I (824) example: Opening file /sdcard/hello.txt I (1114) example: File written speed: 3.59 MB/s I (1134) example: Renaming file /sdcard/hello.txt to /sdcard/foo.txt I (1134) example: Reading file /sdcard/foo.txt I (1134) example: Read from file: 'BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB' I (1144) example: Card unmounted I (1144) main_task: Returned from app_main()

`