Acoustic feedback and echo cancellation problem (Need sound filter) (AUD-2347)

[utkutpcgl]

I use i2s-stream to read the sound detected by the onboard microphone (esp32 Lyrat board) and play the sound with i2s write with an external speaker connected to the onboard speaker output. Also, I only use one channel to listen and play (channel left).

I have the following pipeline: microphone -> i2s_stream_read -> amr_encoder ->amr_decoder ->i2s_stream_write -> speaker (There is approx. 150 ms delay btw. listening and playing the sound due to read-write delays inside the pipeline.)

The microphone listens to the speaker, hence, it enters a loop and repeats to play the previous sound that it has listened to (this is called acoustic feedback). It can even produce a high-intensity noise due to amplification when the output sound volume is set too high. Otherwise, it repeats the sound it listened once, but it is like an undesired echo that I would like to avoid.

I would like to learn how to overcome this problem. It would be good to use a software filter to filter out the speaker's sound, but I could not find the API for this purpose. I see that ESP32-LyraTD-MSC V2.2 has acoustic echo cancellation, is there a way to implement it on ESP32-LyraT (my board)?

Thanks for your help!

Noticed Later I noticed that in an issue algorithm_stream.h was suggested for AEC (acoustic echo cancellation). Do you think it will help me? Also, why isn't there any documentation related to it? Is it a new API?

Thanks again.

[jason-mao]

@utkutpcgl As your description, I think the algo_stream_init could be improve your performance.

[utkutpcgl]

Thanks! Can I use it with ESP32-LyraT?

[jason-mao]

@utkutpcgl Yes. The algorithm stream ALGORITHM_STREAM_INPUT_TYPE2 can be applied.

[utkutpcgl]

Thanks again!

[frj2a]

The "algorithm_stream" that uses "esp_aec" is most probably the way to go. But could not find how to set it up properly. It needs to be inserted in the "input" pipeline but also needs a reference ring-buffer from the "output" pipeline. This last part is the one I could not find out how to set it up.

[Patsjemoe]

Hi, Thank you for your feedback. Found esp_aec.h in components\audiostream\algorithm_stream but cannot find it in the Voip example, despite espressif confirmed it is in it (see attached pictures). Any other example using it? Actually I am not sure how to implement it, so any good suggestion is welcome. Regards Ludo

Op di 13 okt. 2020 19:48 schreef Francisco J. A. Ares < notifications@github.com>:

The "algorithm_stream" that uses "esp_aec" is most probably the way to go. But could not find how to set it up properly. It needs to be inserted in the "input" pipeline but also needs a reference ring-buffer from the "output" pipeline. This last part is the one I could not find out how to set it up.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/espressif/esp-adf/issues/464#issuecomment-707906423, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMMBAJTHCZSJPYQAZWDI54DSKSHF3ANCNFSM4RWTECXA .

[jason-mao]

Similar issue link #476

We will provide a correlative example for demonstrate how to use algorithm_stream.

[Patsjemoe]

Hi, That would be great. Thanks Regards Ludo

Op do 15 okt. 2020 05:42 schreef maojianxin notifications@github.com:

Similar issue link #476 https://github.com/espressif/esp-adf/issues/476

We will provide a correlative example for demonstrate how to use algorithm_stream.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/espressif/esp-adf/issues/464#issuecomment-708876674, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMMBAJW62TGP46GYAL7O6GLSKZVSRANCNFSM4RWTECXA .

[Patsjemoe]

Hi Jason Mao, Any idea when this would come? Espressif documents show it already exists, so I am wondering why I cannot find any example Thank you in advance Regards

[utkutpcgl]

Hello and thanks again @jason-mao , I guess I faced a bug. I would be glad if you could help.

When I tried to build the project with algo stream added I face the following error:

[5/7] Generating ld/esp32.project.ld warning: the default selection ESP32_KORVO_DU1906_CODEC_TAS5805M (undefined) of <choice ESP32_KORVO_DU1906_DAC> (defined at C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/audio_board/Kconfig.projbuild:27) is not contained in the choice [6/7] Linking CXX executable dinleme_deneme.elf FAILED: dinleme_deneme.elf cmd.exe /C "cd . && C:\Users\utkum\.espressif\tools\xtensa-esp32-elf\esp-2020r2-8.2.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-g++.exe -mlongcalls -Wno-frame-address @CMakeFiles\dinleme_deneme.elf.rsp -o dinleme_deneme.elf && cd ." c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o):(.literal.ns_pro+0x48): undefined reference toesp_kiss_fftr_alloc' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o):(.literal.ns_pro+0x4c): undefined reference to esp_kiss_fftr' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o):(.literal.ns_pro+0x58): undefined reference toesp_kiss_fftri' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o): in function ns_noisemean_cal': /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.c:123: undefined reference toesp_kiss_fftr_alloc' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.c:131: undefined reference to esp_kiss_fftr' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o): in functionns_pro': /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.c:217: undefined reference to esp_kiss_fftr' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.c:257: undefined reference toesp_kiss_fftri' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o): in function esp_ns_alloc': /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.h:30: undefined reference toesp_kiss_fftr_alloc' c:/users/utkum/.espressif/tools/xtensa-esp32-elf/esp-2020r2-8.2.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:/Users/utkum/Documents/esp32_idf4.1_latestADF/esp-adf/components/esp-sr/acoustic_algorithm\libesp_audio_processor.a(ns_core.o): in function ns_init': /home/zhaoyang/esp/esp_sr_lib/components/esp_audio_processor/noise_suppression/ns_core.c:295: undefined reference toesp_kiss_fftr_alloc' collect2.exe: error: ld returned 1 exit status ninja: build stopped: subcommand failed. ninja failed with exit code 1`

As far as I understand libesp_audio_processor.a has a reference to esp_kiss_fftr in noise suppression related functions in the ns_core file.

I use:

Latest adf version idf version 4.1 Windows idf command prompt esp32_lyrat board.

Have a nice day.

[Patsjemoe]

The "algorithm_stream" that uses "esp_aec" is most probably the way to go. But could not find how to set it up properly. It needs to be inserted in the "input" pipeline but also needs a reference ring-buffer from the "output" pipeline. This last part is the one I could not find out how to set it up.

Hi, Could you share already how to define the input part? Thanks

[utkutpcgl]

Hello @Patsjemoe , I am not certainly sure but in this pipeline: microphone -> i2s_stream_read -> (input_buffer1) -> amr_encoder ->amr_decoder ->(input_buffer2) ->i2s_stream_write -> speaker I expect the record signal input buffer to be input_buffer1 and the reference signal's buffer to be input_buffer2. If I am wrong please correct me @jason-mao . This seemed logical to me by looking at the following diagram:

Becuase, as far as I know AEC needs an output signal to cancel out the echo, just like @frj2a said.

[jason-mao]

Hi Jason Mao, Any idea when this would come? Espressif documents show it already exists, so I am wondering why I cannot find any example Thank you in advance Regards

@Patsjemoe It will plan to valid on next week.

[utkutpcgl]

Just in case you missed my comment above @jason-mao https://github.com/espressif/esp-adf/issues/464#issuecomment-715164315 . Is this a bug or did I make a mistake? Thanks.

@Patsjemoe Could you find the example code by the way?

Edited: I noticed the link above was not working, now it does.

https://github.com/espressif/esp-adf/issues/494#issue-736790160 -> opened new issue.

[Patsjemoe]

Hi, did not find any example code yet

[Patsjemoe]

Hi Jason Mao, Any idea when this would come? Espressif documents show it already exists, so I am wondering why I cannot find any example Thank you in advance Regards

@Patsjemoe It will plan to valid on next week.

@jason-mao : any progress/idea when it will come? I am trying to find out myself, but the documentation is not sufficiently complete, so more questions then solution

[jason-mao]

@Patsjemoe Yes, we are working on this. There are something need to modify for process normally case.

[Patsjemoe]

OK thank you.

[jason-mao]

@Patsjemoe @frj2a ebb87841016bf0b857a72af119b0d33f6039202b is the algorithm stream example.

[Patsjemoe]

Thank you

[frj2a]

Thank you

[utkutpcgl]

Thanks!