ESP32 Lyra T mini 1.2 ES8311 and Es7243 I2C address and Raw Audio Data (AUD-5636)

[Bhavesh-Moradiya]

Hi, according to schematic ES7243 0x20, but I installed ESP-ADF and successfully tested SD card record audio example lyra T 1.1, in that example I found es7243 i2c address 0x26. How to read raw data from ES7243(Mic Connected) and How to Play Audio from (ES8311 Speaker Connected)? why mini 1.2 has 2 ADC-DAC IC?

Thanks

[Bhavesh-Moradiya]

I want to transmit/recieve data on UDP socket, but I am not familiar with esp32 and i2s.

I did audio board init > i2s0 & i2s1 config > i2s1 channel read > i2s0 channel write

It's similar like echo idf example but it's not working.

[hbler99]

You can try audio_element_input(audio_element_handle_t el, char *buffer, int wanted_size) to obtain audio data from i2s，which is described detailed in https://github.com/espressif/esp-adf/blob/bed20d72b9bbcd48183ef3dddcccb0af3b264eb1/components/audio_pipeline/audio_element.c#L373。 Additionally, there's no need to run the pipeline or element; simply create and initialize the i2s element.

[Bhavesh-Moradiya]

hi hbler99, Thanks.

is it possible to run this audio_element.c on ESP32 Lyra T 1.2? Do I need to initialize audio board before?

Please help me with more detail w.r.t Lyra T 1.2, I am confused with the ES7243 and ES8311 codec chip connected to mic and speaker respectively on Mini 1.2.

Thanks again

[hbler99]

The audio board should be initialized at the beginning. Hope the following steps serve as a helpful reference：

static audio_element_handle_t create_i2s_stream(int sample_rates, int bits, int channels, audio_stream_type_t type)
{
    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT_WITH_PARA(CODEC_ADC_I2S_PORT, sample_rates, bits, type);
    audio_element_handle_t i2s_stream = i2s_stream_init(&i2s_cfg);
    mem_assert(i2s_stream);
    ....
    return i2s_stream;
}

ESP_LOGI(TAG, "[ x ] Create and start input key service");
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);

ESP_LOGI(TAG, "[x] Create i2s stream to write data to codec chip");
audio_element_handle_t i2s_stream_writer = create_i2s_stream(RATE, BITS, CHANNEL, AUDIO_STREAM_WRITER);

ESP_LOGI(TAG, "[x] Create i2s stream to read data from codec chip");
audio_element_handle_t i2s_stream_reader = create_i2s_stream(RATE, BITS, CHANNEL, AUDIO_STREAM_READER);

int read_bytes = audio_element_input(i2s_stream_reader, (void *)block_buffer, size * sizeof(int16_t));

int write_bytes = audio_element_output(i2s_stream_writer, (void *)block_buffer, size * sizeof(int16_t));

[Bhavesh-Moradiya]

Hi @hbler99 Thanks for help

I code as above and received data but its like noise or distortion only, I am not sure about it. may be something I did wrong in buffer size or i2s Initialization. can you check code and correct me what's wrong I did.

static const char *TAG = "I2S RAW DATA";
static esp_periph_set_handle_t set;

#define i2s_ES8311 0
#define i2s_ES7243 1

#define RECORD_RATE 44100
#define RECORD_CHANNEL 1
#define RECORD_BITS 16

#define PLAYBACK_RATE 44100
#define PLAYBACK_CHANNEL 1
#define PLAYBACK_BITS 16

#define EXAMPLE_BUFF_SIZE 2048

static audio_element_handle_t create_i2s_stream(int PORT, int sample_rates, int bits, int channels, audio_stream_type_t type)
{
    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
    i2s_cfg.type = type;
    i2s_cfg.chan_cfg.id = PORT;
    i2s_cfg.transmit_mode = I2S_COMM_MODE_STD;
    i2s_cfg.std_cfg.clk_cfg.sample_rate_hz = sample_rates;
    i2s_cfg.std_cfg.slot_cfg.data_bit_width = bits;
    i2s_cfg.std_cfg.slot_cfg.slot_mode = I2S_SLOT_MODE_MONO;
    i2s_cfg.volume = 90;
    i2s_stream_set_channel_type(&i2s_cfg, I2S_CHANNEL_TYPE_ONLY_RIGHT);
    audio_element_handle_t i2s_stream = i2s_stream_init(&i2s_cfg);
    mem_assert(i2s_stream);

    audio_element_set_music_info(i2s_stream, sample_rates, channels, bits);
    return i2s_stream;
}

static audio_element_handle_t create_wav_encoder()
{
    wav_encoder_cfg_t wav_cfg = DEFAULT_WAV_ENCODER_CONFIG();
    return wav_encoder_init(&wav_cfg);
}

static audio_element_handle_t create_wav_decoder()
{
    wav_decoder_cfg_t wav_cfg = DEFAULT_WAV_DECODER_CONFIG();
    return wav_decoder_init(&wav_cfg);
}

void app_main(void)
{

    uint16_t *w_buf = (uint16_t *)calloc(1, EXAMPLE_BUFF_SIZE);
    assert(w_buf);

    esp_periph_config_t periph_cfg = DEFAULT_ESP_PERIPH_SET_CONFIG();
    set = esp_periph_set_init(&periph_cfg);
    audio_board_key_init(set);

    ESP_LOGI(TAG, "[ x ] Create and start input key service");
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);
    audio_hal_ctrl_codec(board_handle->adc_hal, AUDIO_HAL_CODEC_MODE_ENCODE, AUDIO_HAL_CTRL_START);

    ESP_LOGI(TAG, "[x] Create i2s stream to write data to codec chip");
    audio_element_handle_t i2s_reader_el = create_i2s_stream(i2s_ES7243, RECORD_RATE, RECORD_BITS, RECORD_CHANNEL, AUDIO_STREAM_READER);
    audio_element_handle_t wav_encoder_el = create_wav_encoder();

    ESP_LOGI(TAG, "[x] Create i2s stream to read data from codec chip");
    audio_element_handle_t i2s_writer_el = create_i2s_stream(i2s_ES8311, PLAYBACK_RATE, PLAYBACK_BITS, PLAYBACK_CHANNEL, AUDIO_STREAM_WRITER);
    audio_element_handle_t wav_decoder_el = create_wav_decoder();

    ESP_LOGI(TAG, "[ 3 ] Set up  event listener");
    audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
    audio_event_iface_handle_t evt = audio_event_iface_init(&evt_cfg);

    ESP_LOGI(TAG, "[ 4 ] Set event listening");
    audio_event_iface_set_listener(esp_periph_set_get_event_iface(set), evt);

    while (1)
    {
        ESP_LOGI(TAG, "[ 5 ] Read i2s Data");
        i2s_stream_set_clk(i2s_reader_el, RECORD_RATE, RECORD_BITS, RECORD_CHANNEL);
        int read_bytes = audio_element_input(i2s_reader_el, (int *)w_buf, EXAMPLE_BUFF_SIZE);
    }
}

I did same with pipline and working with it, I can hear echo sound on speaker but I need raw data to transmit it on UDP and vise versa, it will be very helpful if you guide.

for more information here is Log Data from terminal i recorder.

Dat�I (11) boot: ESP-IDF v5.3-dirty 2nd stage bootloader
I (11) boot: compile time Sep  3 2024 03:32:35
I (11) boot: Multicore bootloader
I (15) boot: chip revision: v3.1
I (18) qio_mode: Enabling default flash chip QIO
I (24) boot.esp32: SPI Speed      : 80MHz
I (28) boot.esp32: SPI Mode       : QIO
I (33) boot.esp32: SPI Flash Size : 4MB
I (37) boot: Enabling RNG early entropy source...
I (43) boot: Partition Table:
I (46) boot: ## Label            Usage          Type ST Offset   Length
I (54) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (61) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (69) boot:  2 factory          factory app      00 00 00010000 00100000
I (76) boot: End of partition table
I (80) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=149f4h ( 84468) map
I (110) esp_image: segment 1: paddr=00024a1c vaddr=3ffb0000 size=025f0h (  9712) load
I (113) esp_image: segment 2: paddr=00027014 vaddr=40080000 size=09004h ( 36868) load
I (127) esp_image: segment 3: paddr=00030020 vaddr=400d0020 size=28d58h (167256) map
I (171) esp_image: segment 4: paddr=00058d80 vaddr=40089004 size=09f00h ( 40704) load
I (193) boot: Loaded app from partition at offset 0x10000
I (193) boot: Disabling RNG early entropy source...
I (204) quad_psram: This chip is ESP32-D0WD
I (205) esp_psram: Found 8MB PSRAM device
I (205) esp_psram: Speed: 80MHz
I (206) esp_psram: PSRAM initialized, cache is in low/high (2-core) mode.
W (214) esp_psram: Virtual address not enough for PSRAM, map as much as we can. 4MB is mapped
I (223) cpu_start: Multicore app
I (741) esp_psram: SPI SRAM memory test OK
I (749) cpu_start: Pro cpu start user code
I (749) cpu_start: cpu freq: 240000000 Hz
I (749) app_init: Application information:
I (752) app_init: Project name:     recording_to_sdcard
I (757) app_init: App version:      1
I (762) app_init: Compile time:     Sep  3 2024 03:29:32
I (768) app_init: ELF file SHA256:  d5c1b0d04...
I (773) app_init: ESP-IDF:          v5.3-dirty
I (778) efuse_init: Min chip rev:     v0.0
I (783) efuse_init: Max chip rev:     v3.99 
I (788) efuse_init: Chip rev:         v3.1
I (793) heap_init: Initializing. RAM available for dynamic allocation:
I (800) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (806) heap_init: At 3FFB3228 len 0002CDD8 (179 KiB): DRAM
I (812) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (819) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (825) heap_init: At 40092F04 len 0000D0FC (52 KiB): IRAM
I (831) esp_psram: Adding pool of 4096K of PSRAM memory to heap allocator
I (839) spi_flash: detected chip: gd
I (843) spi_flash: flash io: qio
W (847) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
W (860) ADC: legacy driver is deprecated, please migrate to `esp_adc/adc_oneshot.h`
I (869) main_task: Started on CPU0
I (873) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (881) main_task: Calling app_main()
I (886) AUDIO_THREAD: The button_task task allocate stack on external memory
I (894) AUDIO_THREAD: The esp_periph task allocate stack on internal memory
I (901) I2S RAW DATA: [ x ] Create and start input key service
W (907) i2c_bus_v2: I2C master handle is NULL, will create new one
I (914) gpio: GPIO[18]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0 
I (923) gpio: GPIO[23]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0 
I (938) DRV8311: ES8311 in Slave mode
I (949) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (995) AUDIO_HAL: Codec mode is 3, Ctrl:1
I (1001) AUDIO_HAL: Codec mode is 1, Ctrl:1
I (1001) I2S RAW DATA: [x] Create i2s stream to write data to codec chip
W (1003) file: i2s_stream_idf5.c
I (1008) I2S RAW DATA: [x] Create i2s stream to read data from codec chip
W (1015) file: i2s_stream_idf5.c
I (1019) I2S RAW DATA: [ 3 ] Set up  event listener
I (1024) I2S RAW DATA: [ 4 ] Set event listening
I (1030) I2S RAW DATA: [ 5 ] Read i2s Data
Read Byte: 2048
Data: 64725
Data: 64725
Data: 65500
Data: 44
Data: 65535
Data: 35
Data: 39
Data: 20
Data: 35
Data: 166
Data: 60
Data: 377
Data: 277
Data: 367
Data: 403
Data: 330
Data: 328
Data: 347
Data: 339
Data: 474
Data: 389
Data: 528
Data: 546
Data: 346
Data: 448
Data: 262
Data: 279
Data: 350
Data: 275
Data: 605
Data: 473
Data: 741
Data: 700
Data: 803
Data: 759
Data: 837
Data: 65253
Data: 65387
Data: 65087
Data: 65155
Data: 65016
Data: 65049
Data: 64872
Data: 64959
Data: 64720
Data: 64787
Data: 64615
Data: 64675
Data: 64368
Data: 64502
Data: 64359
Data: 64319

thanks

[hbler99]

I hope the following code will help you.

static const char *TAG = "I2S RAW DATA";
static esp_periph_set_handle_t set;

#define RECORD_RATE 44100
#define RECORD_CHANNEL 1
#define RECORD_BITS 16

#define PLAYBACK_RATE 44100
#define PLAYBACK_CHANNEL 1
#define PLAYBACK_BITS 16

#define EXAMPLE_BUFF_SIZE 512

static audio_element_handle_t create_i2s_stream(int sample_rates, int bits, int channels, audio_stream_type_t type)
{
    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT_WITH_PARA(CODEC_ADC_I2S_PORT, sample_rates, bits, type);
    i2s_cfg.std_cfg.slot_cfg.slot_mode = I2S_SLOT_MODE_MONO;
    i2s_cfg.std_cfg.slot_cfg.slot_mask = I2S_STD_SLOT_LEFT;
    audio_element_handle_t i2s_stream = i2s_stream_init(&i2s_cfg);
    mem_assert(i2s_stream);
    audio_element_set_music_info(i2s_stream, sample_rates, channels, bits);
    return i2s_stream;
}

static audio_element_handle_t create_wav_encoder()
{
    wav_encoder_cfg_t wav_cfg = DEFAULT_WAV_ENCODER_CONFIG();
    return wav_encoder_init(&wav_cfg);
}

static audio_element_handle_t create_wav_decoder()
{
    wav_decoder_cfg_t wav_cfg = DEFAULT_WAV_DECODER_CONFIG();
    return wav_decoder_init(&wav_cfg);
}

void app_main(void)
{

    uint16_t *w_buf = (uint16_t *)audio_calloc(1, EXAMPLE_BUFF_SIZE * sizeof(uint16_t));

    esp_periph_config_t periph_cfg = DEFAULT_ESP_PERIPH_SET_CONFIG();
    set = esp_periph_set_init(&periph_cfg);
    audio_board_key_init(set);

    ESP_LOGI(TAG, "[ x ] Create and start input key service");
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);
    audio_hal_ctrl_codec(board_handle->adc_hal, AUDIO_HAL_CODEC_MODE_ENCODE, AUDIO_HAL_CTRL_START);

    ESP_LOGI(TAG, "[x] Create i2s stream to write data to codec chip");
    audio_element_handle_t i2s_reader_el = create_i2s_stream(RECORD_RATE, RECORD_BITS, RECORD_CHANNEL, AUDIO_STREAM_READER);
    audio_element_handle_t wav_encoder_el = create_wav_encoder();

    ESP_LOGI(TAG, "[x] Create i2s stream to read data from codec chip");
    audio_element_handle_t i2s_writer_el = create_i2s_stream(PLAYBACK_RATE, PLAYBACK_BITS, PLAYBACK_CHANNEL, AUDIO_STREAM_WRITER);
    audio_element_handle_t wav_decoder_el = create_wav_decoder();

    ESP_LOGI(TAG, "[ 3 ] Set up  event listener");
    audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
    audio_event_iface_handle_t evt = audio_event_iface_init(&evt_cfg);

    ESP_LOGI(TAG, "[ 4 ] Set event listening");
    audio_event_iface_set_listener(esp_periph_set_get_event_iface(set), evt);
    ESP_LOGI(TAG, "[ 5 ] Read i2s Data");
    i2s_stream_set_clk(i2s_reader_el, RECORD_RATE, RECORD_BITS, RECORD_CHANNEL);
    while (1)
    {
        int read_bytes = audio_element_input(i2s_reader_el, (void *)w_buf, EXAMPLE_BUFF_SIZE * sizeof(uint16_t));
        int write_bytes = audio_element_output(i2s_writer_el, (void *)w_buf, EXAMPLE_BUFF_SIZE * sizeof(uint16_t));
    }
}

Specially, i2s_stream_set_clk is supposed to be placed outside of the while(1) loop.

[Bhavesh-Moradiya]

@hbler99

Thanks for help brother, its working good.

there is some sound distorted and noisy that need to fix, will check it through "Pipeline" method to get encoded data or resampling.

In code create_i2s_stream function must be initialize separately for both read and write method like below.

   i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
   i2s_cfg.type = type;
   i2s_cfg.chan_cfg.id = port;
   i2s_cfg.std_cfg.slot_cfg.slot_mode = I2S_SLOT_MODE_MONO;
   i2s_stream_set_channel_type(&i2s_cfg, I2S_CHANNEL_TYPE_ONLY_RIGHT);
   audio_element_handle_t i2s_reader_el = i2s_stream_init(&i2s_cfg);
   mem_assert(i2s_reader_el);
   audio_element_set_music_info(i2s_reader_el, 44100, 16, 1);
   i2s_stream_set_clk(i2s_reader_el, 44100, 16, 1);
   return i2s_reader_el;

I achieved my task with I2S > TCP and TCP > I2S using pipeline code and now will do same with new code.

Thanks again for help and support