Environment

Audio development kit: [none]
Audio kit version (for ESP32-LyraT/ESP32-LyraT-Mini/ESP32-S3-Korvo-2): [N/A]
[Required] Module or chip used: [ESP32-S3-WROOM-1]
[Required] IDF version (run git describe --tags in $IDF_PATH folder to find it): v5.1.2
[Required] ADF version (run git describe --tags in $ADF_PATH folder to find it): v2.6-86-g1a03fe6a
Build system: [idf.py]
[Required] Running log: All logs from power-on to problem recurrence
Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it): xtensa-esp32s3-elf-gcc.exe (crosstool-NG esp-12.2.0_20230208) 12.2.0
Operating system: [Windows]
(Windows only) Environment type: [ESP Command Prompt]
Using an IDE?: [Yes (VS Code with ESP-IDF extension)]
Power supply: [USB]

Problem Description

Hi. I'm out of options and desperately need some help. Maybe somebody had this problem before. Basically, I'm just trying to use the Espressif's Audio Forge example(https://github.com/espressif/esp-adf/blob/master/examples/audio_processing/pipeline_audio_forge/main/audio_forge_pipeline_main.c) from ESP-ADF repository and compile it for my custom ESP32-S3 board. Everything works, and I CAN HEAR sound from the speaker when I

#define NUMBER_SOURCE_FILE 1,

but it's showing me the

"AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set"

when I #define NUMBER_SOURCE_FILE 2. The modification I've done was to add for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {esp_err_t ret = audio_pipeline_run(pipeline[i]);} audio_pipeline_run(pipeline_mix); before while (1) {} to start the audio pipeline because I don't have a board with the buttons and use MP3 instead of WAV.

This is my console output:

Expected Behavior

I should hear 2 sounds simultaneously.

Actual Behavior

E (1546) AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set

And silence...

Steps to Reproduce

define NUMBER_SOURCE_FILE 2
Compile and run.

20240404_154555

I (1436) DS1302: [3.0] Create pipeline_mix to mix I (1436) DS1302: [3.1] Create audio_forge I (1436) DS1302: [3.2] Create i2s stream to read audio data from codec chip I (1436) DS1302: [3.3] Link elements together audio_forge-->i2s_writer I (1436) DS1302: [3.4] Link elements together audio_forge-->i2s_stream-->[codec_chip] I (1436) DS1302: [4.0] Create Fatfs stream to read input data I (1436) DS1302: [4.1] Create wav decoder to decode wav file I (1436) DS1302: [4.2] Create raw stream of base wav to write data I (1436) DS1302: [5.0] Set up event listener I (1436) MP3_DECODER: MP3 init I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c077d78, tag:file, rb:0x3c0781b0 I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c077f10, tag:wav, rb:0x3c07a1f8 I (1436) MP3_DECODER: MP3 init I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c07aaa8, tag:file, rb:0x3c07aee0 I (1446) gpio: GPIO[0]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3 I (1446) gpio: GPIO[1]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3 I (1446) AUDIO_PIPELINE: link el->rb, el:0x3c07ac40, tag:wav, rb:0x3c07cf28 I (1446) DS1302: [5.1] Listening event from peripherals I (1446) AUDIO_THREAD: The file task allocate stack on internal memory I (1446) AUDIO_ELEMENT: [file-0x3c077d78] Element task created I (1446) AUDIO_THREAD: The wav task allocate stack on external memory I (1446) AUDIO_ELEMENT: [wav-0x3c077f10] Element task created I (1446) AUDIO_ELEMENT: [raw-0x3c078040] Element task created I (1446) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8634896 Bytes, Inter:340923 Bytes, Dram:340923 Bytes

I (1446) AUDIO_ELEMENT: [file] AEL_MSG_CMD_RESUME,state:1 I (1446) AUDIO_ELEMENT: [wav] AEL_MSG_CMD_RESUME,state:1 I (1446) MP3_DECODER: MP3 opened I (1446) FATFS_STREAM: File size: 767885 byte, file position: 0 I (1446) AUDIO_PIPELINE: Pipeline started I (1446) AUDIO_THREAD: The file task allocate stack on internal memory I (1446) AUDIO_ELEMENT: [file-0x3c07aaa8] Element task created I (1446) AUDIO_THREAD: The wav task allocate stack on external memory I (1456) AUDIO_ELEMENT: [wav-0x3c07ac40] Element task created I (1456) AUDIO_ELEMENT: [raw-0x3c07ad70] Element task created I (1456) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8618684 Bytes, Inter:335531 Bytes, Dram:335531 Bytes

I (1456) AUDIO_ELEMENT: [file] AEL_MSG_CMD_RESUME,state:1 I (1456) AUDIO_ELEMENT: [wav] AEL_MSG_CMD_RESUME,state:1 I (1456) MP3_DECODER: MP3 opened I (1456) AUDIO_PIPELINE: Pipeline started I (1456) AUDIO_THREAD: The audio_forge task allocate stack on external memory I (1456) CODEC_ELEMENT_HELPER: The element is 0x3c077f10. The reserve data 2 is 0x0. I (1466) AUDIO_ELEMENT: [audio_forge-0x3c0775e8] Element task created I (1466) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8575400 Bytes, Inter:335703 Bytes, Dram:335703 Bytes

I (1466) AUDIO_ELEMENT: [audio_forge] AEL_MSG_CMD_RESUME,state:1 I (1466) AUDIO_FORGE: audio_forge opened I (1466) AUDIO_PIPELINE: Pipeline started W (1466) DS1302: [ ] Receive music info from wav decoder, sample_rates=44100, bits=16, ch=1 I (1476) FATFS_STREAM: File size: 736584 byte, file position: 0 I (1486) CODEC_ELEMENT_HELPER: The element is 0x3c07ac40. The reserve data 2 is 0x0. W (1526) DS1302: [ ] Receive music info from wav decoder, sample_rates=44100, bits=16, ch=1 E (1546) AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set I (1546) AUDIO_FORGE: audio forge closed I (1546) DS1302: [6.0] Stop pipelines W (1546) AUDIO_ELEMENT: OUT-[wav] AEL_IO_ABORT W (1546) MP3_DECODER: Output aborted -3 I (1546) MP3_DECODER: Closed W (1546) AUDIO_ELEMENT: OUT-[file] AEL_IO_ABORT W (1556) AUDIO_ELEMENT: OUT-[wav] AEL_IO_ABORT W (1556) MP3_DECODER: Output aborted -3 I (1556) MP3_DECODER: Closed W (1556) AUDIO_ELEMENT: OUT-[file] AEL_IO_ABORT W (1556) AUDIO_ELEMENT: [audio_forge] Element already stopped W (1556) AUDIO_PIPELINE: There are no listener registered I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked W (1556) AUDIO_ELEMENT: [file] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_ELEMENT: [wav] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_ELEMENT: [raw] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_PIPELINE: There are no listener registered I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked W (1556) AUDIO_ELEMENT: [file] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_ELEMENT: [wav] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_ELEMENT: [raw] Element has not create when AUDIO_ELEMENT_TERMINATE W (1556) AUDIO_PIPELINE: There are no listener registered I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked W (1556) AUDIO_ELEMENT: [audio_forge] Element has not create when AUDIO_ELEMENT_TERMINATE I (1556) AUDIO_FORGE: audio_forge_destroy I (1556) AUDIO_FORGE: Func:audio_forge_destroy, Line:406, MEM Total:8674748 Bytes, Inter:349959 Bytes, Dram:349959 Bytes

W (1556) AUDIO_ELEMENT: [iis] Element has not create when AUDIO_ELEMENT_TERMINATE



## Other Items If Possible

- [ ] sdkconfig file (Attach the sdkconfig file from your project folder)
[sdkconfig.txt](https://github.com/espressif/esp-adf/files/14872947/sdkconfig.txt)

Hi, this issue is already fixed. If you only need mix, you can use example audio_mixer_tone. If you must use audio forege, for your needs, you can use this test code ` /* Multiple pipeline playback with audio processing.

This example code is in the Public Domain (or CC0 licensed, at your option.)

Unless required by applicable law or agreed to in writing, this software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. */

include

include "freertos/FreeRTOS.h"

include "freertos/task.h"

include "freertos/event_groups.h"

include "esp_log.h"

include "audio_element.h"

include "audio_pipeline.h"

include "audio_event_iface.h"

include "i2s_stream.h"

include "mp3_decoder.h"

include "fatfs_stream.h"

include "audio_forge.h"

include "raw_stream.h"

include "board.h"

include "periph_button.h"

static const char *TAG = "AUDIO_FORGE_PIPELINE";

define DEFAULT_SAMPLERATE 44100

define DEFAULT_CHANNEL 2

define DEST_SAMPLERATE 11025

define DEST_CHANNEL 1

define TRANSMITTIME 0

define MUSIC_GAIN_DB -10

define NUMBER_SOURCE_FILE 2

int audio_forge_wr_cb(audio_element_handle_t el, char buf, int len, TickType_t wait_time, void ctx) { audio_element_handle_t i2s_wr = (audio_element_handle_t)ctx; int ret = audio_element_output(i2s_wr, buf, len); return ret; }

void app_main(void) { audio_pipeline_handle_t pipeline[NUMBER_SOURCE_FILE] = {NULL}; audio_element_handle_t fats_rd_el[NUMBER_SOURCE_FILE] = {NULL}; audio_element_handle_t mp3_decoder[NUMBER_SOURCE_FILE] = {NULL}; audio_element_handle_t el_raw_write[NUMBER_SOURCE_FILE] = {NULL};

esp_log_level_set("*", ESP_LOG_INFO);
esp_log_level_set(TAG, ESP_LOG_INFO);

ESP_LOGI(TAG, "[1.0] Start audio codec chip");
audio_board_handle_t board_handle = audio_board_init();
audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_DECODE, AUDIO_HAL_CTRL_START);

ESP_LOGI(TAG, "[2.0] Start and wait for SDCARD to mount");
esp_periph_config_t periph_cfg = DEFAULT_ESP_PERIPH_SET_CONFIG();
esp_periph_set_handle_t set = esp_periph_set_init(&periph_cfg);
audio_board_sdcard_init(set, SD_MODE_1_LINE);
audio_board_key_init(set);

ESP_LOGI(TAG, "[3.0] Create pipeline_mix to mix");
audio_pipeline_cfg_t pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG();
audio_pipeline_handle_t pipeline_mix = audio_pipeline_init(&pipeline_cfg);

ESP_LOGI(TAG, "[3.1] Create audio_forge");
audio_forge_cfg_t audio_forge_cfg = AUDIO_FORGE_CFG_DEFAULT();
audio_forge_cfg.audio_forge.component_select = AUDIO_FORGE_SELECT_RESAMPLE | AUDIO_FORGE_SELECT_DOWNMIX | AUDIO_FORGE_SELECT_ALC | AUDIO_FORGE_SELECT_EQUALIZER | AUDIO_FORGE_SELECT_SONIC;
audio_forge_cfg.audio_forge.dest_samplerate = DEST_SAMPLERATE;
audio_forge_cfg.audio_forge.dest_channel = DEST_CHANNEL;
audio_forge_cfg.audio_forge.source_num = NUMBER_SOURCE_FILE;
audio_forge_cfg.audio_forge.max_sample = 128;
audio_forge_cfg.task_prio = 4;
audio_element_handle_t audio_forge = audio_forge_init(&audio_forge_cfg);
audio_forge_src_info_t source_information = {
    .samplerate = DEFAULT_SAMPLERATE,
    .channel = DEFAULT_CHANNEL,
    .bit_num = 16,
};

audio_forge_downmix_t downmix_information = {
    .gain = {0, MUSIC_GAIN_DB},
    .transit_time = TRANSMITTIME,
};
audio_forge_src_info_t source_info[NUMBER_SOURCE_FILE] = {0};
audio_forge_downmix_t downmix_info[NUMBER_SOURCE_FILE];
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    source_info[i] = source_information;
    downmix_info[i] = downmix_information;
}
audio_forge_source_info_init(audio_forge, source_info, downmix_info);

ESP_LOGI(TAG, "[3.2] Create i2s stream to read audio data from codec chip");
i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
i2s_cfg.type = AUDIO_STREAM_WRITER;
i2s_cfg.task_stack = 0;
i2s_cfg.out_rb_size = 0;

if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)

i2s_cfg.chan_cfg.auto_clear = true;

else

i2s_cfg.i2s_config.tx_desc_auto_clear = true;

endif

audio_element_handle_t i2s_writer = i2s_stream_init(&i2s_cfg);
i2s_stream_set_clk(i2s_writer, DEST_SAMPLERATE, 16, DEST_CHANNEL);

ESP_LOGI(TAG, "[3.3] Link elements together audio_forge-->i2s_writer");
audio_pipeline_register(pipeline_mix, audio_forge, "audio_forge");
audio_element_set_write_cb(audio_forge, audio_forge_wr_cb, i2s_writer);
audio_element_process_init(i2s_writer);

ESP_LOGI(TAG, "[3.4] Link elements together audio_forge-->i2s_stream-->[codec_chip]");
audio_pipeline_link(pipeline_mix, (const char *[]) {"audio_forge"}, 1);

ESP_LOGI(TAG, "[4.0] Create Fatfs stream to read input data");
fatfs_stream_cfg_t fatfs_cfg = FATFS_STREAM_CFG_DEFAULT();
fatfs_cfg.type = AUDIO_STREAM_READER;

ESP_LOGI(TAG, "[4.1] Create mp3 decoder to decode mp3 file");
mp3_decoder_cfg_t mp3_cfg = DEFAULT_MP3_DECODER_CONFIG();
mp3_cfg.task_core = 0;

ESP_LOGI(TAG, "[4.2] Create raw stream of base mp3 to write data");
raw_stream_cfg_t raw_cfg = RAW_STREAM_CFG_DEFAULT();
raw_cfg.type = AUDIO_STREAM_WRITER;

ESP_LOGI(TAG, "[5.0] Set up  event listener");
audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
audio_event_iface_handle_t evt = audio_event_iface_init(&evt_cfg);

char str_name[18] = "/sdcard/test";
char num = '1';
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    pipeline[i] = audio_pipeline_init(&pipeline_cfg);
    mem_assert(pipeline[i]);
    fats_rd_el[i] = fatfs_stream_init(&fatfs_cfg);
    str_name[12] = num + i;
    str_name[13] = '.';
    str_name[14] = 'm';
    str_name[15] = 'p';
    str_name[16] = '3';
    audio_element_set_uri(fats_rd_el[i], str_name);
    mp3_cfg.task_core = i; // mp3 decoder set to differen cpu core
    // mp3_cfg.task_prio = 12 - i - i;
    mp3_decoder[i] = mp3_decoder_init(&mp3_cfg);
    el_raw_write[i] = raw_stream_init(&raw_cfg);
    audio_pipeline_register(pipeline[i], fats_rd_el[i], "file");
    audio_pipeline_register(pipeline[i], mp3_decoder[i], "mp3");
    audio_pipeline_register(pipeline[i], el_raw_write[i], "raw");

    const char *link_tag[3] = {"file", "mp3", "raw"};
    audio_pipeline_link(pipeline[i], &link_tag[0], 3);
    ringbuf_handle_t rb = audio_element_get_input_ringbuf(el_raw_write[i]);
    if (NUMBER_SOURCE_FILE != 1) {
        audio_element_set_multi_input_ringbuf(audio_forge, rb, i);
    } else {
        audio_element_set_input_ringbuf(audio_forge, rb);
    }
    audio_pipeline_set_listener(pipeline[i], evt);
}
audio_pipeline_set_listener(pipeline_mix, evt);
ESP_LOGI(TAG, "[5.1] Listening event from peripherals");
audio_event_iface_set_listener(esp_periph_set_get_event_iface(set), evt);
audio_forge_downmix_set_transit_time(audio_forge, 100, 0);
audio_forge_downmix_set_transit_time(audio_forge, 100, 1);
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    audio_pipeline_run(pipeline[i]);
}
audio_pipeline_run(pipeline_mix);
while (1) {
    audio_event_iface_msg_t msg;
    esp_err_t ret = audio_event_iface_listen(evt, &msg, portMAX_DELAY);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "[ * ] Event interface error : %d", ret);
        continue;
    }

    for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT && msg.source == (void *)mp3_decoder[i]
            && msg.cmd == AEL_MSG_CMD_REPORT_MUSIC_INFO) {
            audio_element_info_t music_info = {0};
            audio_element_getinfo(mp3_decoder[i], &music_info);
            ESP_LOGW(TAG, "[ * ] Receive music info from wav decoder, sample_rates=%d, bits=%d, ch=%d",
                     music_info.sample_rates, music_info.bits, music_info.channels);
            audio_forge_src_info_t src_info = {
                .samplerate = music_info.sample_rates,
                .channel = music_info.channels,
                .bit_num = music_info.bits,
            };
            audio_forge_set_src_info(audio_forge, src_info, i);
        }
    }
    /* Stop when the last pipeline element (fatfs_writer in this case) receives stop event */
    if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT
        && msg.source == (void *) audio_forge
        && msg.cmd == AEL_MSG_CMD_REPORT_STATUS
        && (((int)msg.data == AEL_STATUS_STATE_STOPPED)
            || ((int)msg.data == AEL_STATUS_STATE_FINISHED))) {
        break;
    }
}

ESP_LOGI(TAG, "[6.0] Stop pipelines");
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    audio_pipeline_stop(pipeline[i]);
    audio_pipeline_wait_for_stop(pipeline[i]);
    audio_pipeline_terminate(pipeline[i]);
    audio_pipeline_unregister_more(pipeline[i], fats_rd_el[i], mp3_decoder[i], el_raw_write[i], NULL);
    audio_pipeline_remove_listener(pipeline[i]);
}
audio_pipeline_stop(pipeline_mix);
audio_pipeline_wait_for_stop(pipeline_mix);
audio_pipeline_terminate(pipeline_mix);
audio_pipeline_unregister_more(pipeline_mix, audio_forge, NULL);
audio_pipeline_remove_listener(pipeline_mix);

/* Stop all peripherals before removing the listener */
esp_periph_set_stop_all(set);
audio_event_iface_remove_listener(esp_periph_set_get_event_iface(set), evt);

/* Make sure audio_pipeline_remove_listener & audio_event_iface_remove_listener are called before destroying event_iface */
audio_event_iface_destroy(evt);

for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    /* Release resources */
    audio_pipeline_deinit(pipeline[i]);
    audio_element_deinit(fats_rd_el[i]);
    audio_element_deinit(mp3_decoder[i]);
    audio_element_deinit(el_raw_write[i]);
}
/* Release resources */
audio_pipeline_deinit(pipeline_mix);
audio_element_deinit(audio_forge);
audio_element_deinit(i2s_writer);
esp_periph_set_destroy(set);

} `

espressif / esp-adf

AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

Environment

Problem Description

Expected Behavior

Actual Behavior

Steps to Reproduce

define NUMBER_SOURCE_FILE 2

include

include "freertos/FreeRTOS.h"

include "freertos/task.h"

include "freertos/event_groups.h"

include "esp_log.h"

include "audio_element.h"

include "audio_pipeline.h"

include "audio_event_iface.h"

include "i2s_stream.h"

include "mp3_decoder.h"

include "fatfs_stream.h"

include "audio_forge.h"

include "raw_stream.h"

include "board.h"

include "periph_button.h"

define DEFAULT_SAMPLERATE 44100

define DEFAULT_CHANNEL 2

define DEST_SAMPLERATE 11025

define DEST_CHANNEL 1

define TRANSMITTIME 0

define MUSIC_GAIN_DB -10

define NUMBER_SOURCE_FILE 2

if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)

else

endif