Unable to use filter_resample to resample in small increments (AUD-5054)

ftab commented 7 months ago

Environment

Audio development kit: Custom board: ESP32 -> DAC -> amps. Running at nominally 44.1khz, receiving A2DP music.
[Required] Module or chip used: ESP32-WROVER-E
[Required] IDF version (run git describe --tags in $IDF_PATH folder to find it): // v5.0.3-147-gdc9a3c43ea - my fork https://github.com/radiosound-com/esp-idf/tree/release/v5.0+0634bb9+adfpatch
[Required] ADF version (run git describe --tags in $ADF_PATH folder to find it): // v2.5-76-g6207836f - my fork https://github.com/radiosound-com/esp-adf/tree/radiosound-modded-v2.5-66-g49e63cd5
Build system: idf.py
Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it): // xtensa-esp32-elf-gcc (crosstool-NG esp-2022r1) 11.2.0
Operating system: Linux
Using an IDE?: vscode or none
Power supply: Custom board with benchtop power supply set to 12V. Board has 3.3V regulator to power ESP32.

Problem Description

I am using a pipeline with a2dp_stream -> filter_resample -> dsp (my own code) -> i2s and trying to adjust for differences in clocks / playback rate. I am trying to work around https://github.com/espressif/esp-idf/issues/11688#issuecomment-1830316668

Initial config:

    rsp_filter_cfg_t rsp_d_cfg = {
        .src_rate = 44100,
        .src_ch = 2,
        .src_bits = 16,
        .dest_rate = 44100,
        .dest_ch = 2,
        .dest_bits = 16,
        .mode = RESAMPLE_ENCODE_MODE,
        .max_indata_bytes = RSP_FILTER_BUFFER_BYTE,
        .out_len_bytes = 2048,
        .type = ESP_RESAMPLE_TYPE_AUTO,
        .complexity = 2,
        .down_ch_idx = 0,
        .prefer_flag = ESP_RSP_PREFER_TYPE_SPEED,
        .out_rb_size = 8192,
        .task_stack = RSP_FILTER_TASK_STACK,
        .task_core = RSP_FILTER_TASK_CORE,
        .task_prio = RSP_FILTER_TASK_PRIO,
        .stack_in_ext = true,
    };
    filter_d = rsp_filter_init(&rsp_d_cfg);

I have a thread that estimates the actual src playback rate (by looking at the output ringbuffer filled percentage average, sampled every 10ms for 500ms), and then for example, after determining that the buffer is getting too full and it should play faster (stream_sample_adj = -1) calls

rsp_filter_change_src_info(filter_d, 44100 + (stream_sample_adj << 2), 2, 16);

The intent is to make minor adjustments in sample rate (create a small number of samples to play slower, destroy a small number of samples to play faster) to account for differences in source device's clock vs. my clock.

Expected Behavior

Resample filter should take small adjustments

Actual Behavior

I (12741) adj: History [40, 40, 65]
I (12746) adj: play faster
I (12748) adj: new offset = -4 hz
I (12757) RSP_FILTER: Reset sample rate of source data : 44096, reset channel of source data : 2, reset bit per sample of source data : 16
E (12766) FIR_RESAMPLE: sample rate is out of range [8000 96000], set input samplerate = 44096 : output sample rate = 44100
E (12772) RSP_FILTER: Failed to create the resample handler
E (12774) AUDIO_ELEMENT: [filter_d] ERROR_PROCESS, AEL_PROCESS_FAIL
W (12777) AUDIO_ELEMENT: [filter_d] audio_element_on_cmd_error,3

Steps to Reproduce

Currently unable to share code / steps as this is all on a custom board.

If I can reproduce on LyraT I will share that. But I figured I would ask first to see if the resample filter is even supposed to be able to handle this sort of small adjustment. It would seem like it should, but I'm often wrong.

TempoTian commented 7 months ago

Resample only support fixed sample rate like (8k, 16k, 44.1, 48k) and not support other rate which not in it You can try to use sonic module if the sample rate does not change too much.

void esp_sonic_set_speed(void* handle, float speed);

Under path: https://github.com/espressif/esp-adf-libs/blob/619b3bac9a34617f288c6e68f3c587618b1c68bb/esp_codec/include/processing/esp_sonic.h

ftab commented 7 months ago

Thanks for the info. I have dusted off some of my old code that does the small sample rate adjustment and ported it to esp-adf, so I have solved the problem. But thank you for pointing out esp_sonic as well.

espressif / esp-adf