downmix volume transition: volume drop and clicking (AUD-3356)

[stwirth]

I'm using downmix to create an infinite loop of the same sound by playing a WAV file twice and transition between both smoothly.

I'm using a simple 440Hz tone like this for testing.

My problem is now that I can hear a volume decrease and also some clicking during the transition. Is the gain of both inputs increased and decreased linearly? I'd like to have a constant power cross-fade like this instead: Is there a way to change the type of fade? Any idea on how I can avoid the clicking?

The full source code of my test can be found here: https://gist.github.com/stwirth/367fba0c07dd7bd3567be11dcf889c48

I'm using the ESP32-LyraT V4.3 with and up to date esp-adf from the master branch.

[houhaiyan]

@stwirth hello, the gain increase/decrease is linearly. Nonlinear is unsupported. Maybe the clicking is because any channel stream is blocked before down-mix. You can increase ring-buffer waiting data time using downmix_set_input_rb_timeout. If the clicking is still happened, please save the stream after down-mix. Then send to me.

[stwirth]

@houhaiyan thanks for the reply!

I set the timeout to

downmix_set_input_rb_timeout(downmixer, 250, INDEX_BASE_STREAM);
downmix_set_input_rb_timeout(downmixer, 250, INDEX_NEWCOME_STREAM);

but I can still hear the clicking.

I tried to set up a pipeline to write to a WAV file while mixing but the resulting file is always empty. My attempt is here: https://gist.github.com/stwirth/367fba0c07dd7bd3567be11dcf889c48#file-main_write-c

[houhaiyan]

@stwirth Maybe the clicking is because two pipeline reading the one file, file system will be blocked. .

1. copy "/sdcard/440Hz_44100Hz_16bit_05sec.wav" to "/sdcard/440Hz_44100Hz_16bit_05sec_copy.wav"

2. audio_element_set_uri(base_fatfs_reader_el, "/sdcard/440Hz_44100Hz_16bit_05sec.wav"); ->audio_element_set_uri(base_fatfs_reader_el, "/sdcard/440Hz_44100Hz_16bit_05sec_copy.wav"); If the clicking is still happened, please upload the other channel stream. I will re-check the code.

   For you can't save the mix stream, you can try again after modifing as the follow.

   ESP_LOGI(TAG, "[3.2] Create i2s stream to read audio data from codec chip");
   i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
   i2s_cfg.type = AUDIO_STREAM_WRITER;
   audio_element_handle_t i2s_writer = i2s_stream_init(&i2s_cfg);
   
   ESP_LOGI(TAG, "[3.3] Link elements together downmixer-->i2s_writer");
   audio_pipeline_register(pipeline_mix, downmixer, "mixer");
   audio_pipeline_register(pipeline_mix, i2s_writer, "i2s");

   ->

   ESP_LOGI(TAG, "[3.2] Create i2s stream to read audio data from codec chip");
   fatfs_stream_cfg_t fatfs_write_cfg = FATFS_STREAM_CFG_DEFAULT();
   fatfs_write_cfg.type = AUDIO_STREAM_WRITER;
   audio_element_handle_t el_fatfs_wr_stream = fatfs_stream_init(&fatfs_write_cfg);
   
   ESP_LOGI(TAG, "[3.3] Link elements together downmixer-->file system");
   audio_pipeline_register(pipeline_mix, downmixer, "mixer");
   audio_pipeline_register(pipeline_mix, el_fatfs_wr_stream, "fatsf");
   audio_element_set_uri(el_fatfs_wr_stream, "/sdcard/rec.wav");

   The stream after down-mix can be recorded. But you can't hear from speaker . If you want to hear and save. please refer in https://github.com/espressif/esp-adf/tree/master/examples/recorder/element_wav_amr_sdcard

[stwirth]

@houhaiyan thanks for your hints. Even when I use two separate wav files I can hear the clicking.

I would like to send you the output as wav but I can't get the fatfs writer to work. It seems that my logic to switch between base and newcome track will not run before the tracks finish and the output file is empty (my attempt is in main_write2.c in https://gist.github.com/stwirth/367fba0c07dd7bd3567be11dcf889c48#file-main_write2-c). Anyhow, if you have a LyraT V4.3 you should be able to reproduce the issue easily with the source code I provided above.

[stwirth]

To achieve the non-linear volume change, is there a way to modify the volume of a single audio element? I'm thinking of using a downmix to mix the two input streams but modify the gain of the input streams before they enter the downmix element somehow.

[stwirth]

Maybe I'm making this too complicated. All I want is a click-free infinite loop of the same sound. Is there a way to tell the fatfs stream reader to start from the beginning of the file again when it gets to the end?

[houhaiyan]

@stwirth
test.zip I do some modification for https://gist.github.com/stwirth/367fba0c07dd7bd3567be11dcf889c48#file-main_write2-c) I don't find the clicking

[jason-mao]

This topic has become inactive so I'm going to close the issue. Please reopen this if you have any questions or need any further assistance.