search

luxai-qtrobot / QA

Virtual repository for Questions & Answers system
http://luxai-qtrobot.github.io
5 stars 0 forks source link

Acoustic Echo Cancellation #48

Open TechnoX opened 3 years ago

TechnoX commented 3 years ago

I'm investigating the ReSpeaker and it has Acoustic Echo Cancellation (AEC) that seems to work quite well if I look at their videos, such as https://www.youtube.com/watch?v=XivM-6PRgX8

However it is only possible to use the AEC if you use the ReSpeaker as an output device, and connect the speakers through the ReSpeaker. But I suspect the speakers aren't connected to the ReSpeaker within QT?

The reason I want to use AEC is to prevent the robot from hearing itself and also to play some other sound files. It is of course possible to block the hearing when the robot is speaking. But I would like to be able to talk to the robot at the same time itself talks, for example to say "Stop talking" to it.

apaikan commented 3 years ago

Correct. The speakers of QTrobot are not connected to ReSpeaker. Instead, they are connected to an integrated amplifier through routed audio signal from RP for better EMI compatibility.

Doesn't a hotword detector do the job? somethin like Hey QT Stop talking!. Usually hotword detectors are less sensible to other noises.

TechnoX commented 3 years ago

Unfortunately, I don't think a hotword detector will make it. When the robot speaks (at least on the volume we have, I don't remember the exact value) its almost impossible to hear anything from the microphones. I don't know if it's only for us or if this is the situation to deal with. The ReSpeaker is mounted quite close to the speakers I suspect. If you want i can record some example audio files when I'm talking to the robot and it talks to me.

Another issue is the fan that creates quite loudly noise on the microphone. But I have been able to remove almost all noise by adjusting ReSpeakers parameters. If anyone searching online for good values finds this post, this was what I came up with: 

        self.dev.write("AGCONOFF", 1)
        self.dev.write("AGCGAIN", 13.0)
        self.dev.write("CNIONOFF", 0)
        self.dev.write("AGCDESIREDLEVEL", 0.5)
        self.dev.write("MIN_NS_SR", 0.0001)
        self.dev.write("GAMMA_NS_SR", 3.0)

Maybe you at luxAI have found even better values for suppressing the noise from the fan?

Regarding Acoustic Echo Cancellation I tried to load a loopback module from the integrated amplifier to the ReSpeaker. This was the config for it: 

qtrobot@QTRD000185:~$ pactl list short sinks
0   alsa_output.usb-SEEED_ReSpeaker_4_Mic_Array__UAC1.0_-00.analog-stereo   module-alsa-card.c  s24le 2ch 16000Hz   IDLE
1   alsa_output.platform-soc_audio.analog-stereo    module-alsa-card.c  s16le 2ch 44100Hz   IDLE
qtrobot@QTRD000185:~$ pactl list short sources
0   alsa_output.usb-SEEED_ReSpeaker_4_Mic_Array__UAC1.0_-00.analog-stereo.monitor   module-alsa-card.c  s24le 2ch 16000Hz   SUSPENDED
1   alsa_input.usb-SEEED_ReSpeaker_4_Mic_Array__UAC1.0_-00.multichannel-input   module-alsa-card.c  s16le 6ch 16000Hz   SUSPENDED
2   alsa_output.platform-soc_audio.analog-stereo.monitor    module-alsa-card.c  s16le 2ch 44100Hz   SUSPENDED
qtrobot@QTRD000185:~$ pactl load-module module-loopback sink=0 source=2
23

But the time difference between the sound wave detected by the microphone and the sound directed through the loopback to the ReSpeaker is too large so it doesn't remove anything that the robot is saying. Maybe it is possible to adjust the parameters of the ReSpeaker to compensate for the larger time difference, but I haven't find any settings that work.

What I can see is that the channel 5 of the ReSpeaker is filled with sound-data so at least it is trying to compute the echo and saves it on the right channel. So I know that my loopback module is working as expected.

apaikan commented 3 years ago

Really nice and interesting work @TechnoX

Unfortunately we have not yet exploited the Respeaker to its best features. I would really like to get some time and try its feature through developing some demos/tutorials too. Recently we have installed and implemented ROS interface for VOSK offline speech recognition. It's working better than we were expected especially when an external Respeaker mic is used.

Fan Noise At very beginning, when designing the robot head and placing the mic, we did some acoustic test to find the right place for it in the head and the cuts/holes in the head for the microphone. Apparently with some updated version of NUC and fan, some more noise are introduced to the system. Anyway, we will definitely try your configuration for the noise canceling and will share your finding in our documentation.

Echo cancelation In our new design of the robot, we have already considered the possibility of using Respeaker audio output for the robot instead of RPI built in. However for now,

apaikan commented 3 years ago

Hi @TechnoX Recently we got a bit of time to work on the Respeaker tuning for voice recognition using VOSK. We got pretty much close to the same value you have proposed for background noise cancelation with disabling automatic gain control (as also recommended by Google Speech recognition best practices).

Anyway to facilitate the tuning, we have developed a simple GUI which might be interested for you too.

image

feel free to try it from here respeaker_mic_tunning

Cheers, Ali