Manually cut echo: deafen myself or mute others when talking

[kaddkaka]

Context In some scenarios we have a bad system set up with loud speakers and sensitive microphones in both ends.

1. On one end it's an open microphone in an office room that is supposed to be useable for anyone in the room (so not possible to use push-to-talk on that end).
2. In the other end there is a laptop with loudspeakers, builtin microphone and sometimes long latency connections ( push-to-talk can be used on this end).

In these situations talking from the laptop (2) will result in a lot of echo and it's very difficult to communicate.

Describe the feature you have in mind Describe alternatives you've considered I have a few different ideas to solve this:

In the room (1):

1.A. Mute self when others are speaking 1.B. Change Audio in sensitivity when others are speaking

On the laptop (2):

2.A. Deafen self when talking (push-to-talk-and-deafen) 2.B. Mute others when talking (push-to-talk-and-mute-others)

I'm not sure which of these would be the best, but I'm leaning to 1A as the one that would solve my particular issue in the best way.

[Krzmbrzl]

Mute self when others are speaking

I think this is something that I have already seen in e.g. Zoom. While I find it a bit annoying, it is probably still better than having to deal with massive echo :thinking:

[kaddkaka]

Mute self when others are speaking

I think this is something that I have already seen in e.g. Zoom. While I find it a bit annoying, it is probably still better than having to deal with massive echo 🤔

Yeah, it's not perfect, but better than echo and also better than hearing oneself.

[jaggzh]

It's this one agreed-upon as a good feature? (Whether anyone's available to implement it is another story)

[Krzmbrzl]

Yes - otherwise the issue would have been closed :point_up:

[jaggzh]

I think I accidentally added it to a generated .h file

[Krzmbrzl]

Hm? That looks as if you added it in Qt designer while editing the ui file

[jaggzh]

I just did it by hand, not qt designer. I'm working on it here, but obviously don't know what I'm doing :) https://github.com/mumble-voip/mumble/compare/master...jaggzh:mumble:talk-deafens-self

[Krzmbrzl]

I would strongly advise against editing ui files by hand

[davidebeatrici]

I would only do it when the diff becomes cluttered, due to Qt Designer moving blocks around for example.

[Krzmbrzl]

Even then. Chances are too high to screw things up...

[jaggzh]

src/murmur/Messages.cpp

    if (!qvSuggestPushToTalk.isNull())
        mpsug.set_push_to_talk(qvSuggestPushToTalk.toBool());
        mpus.set_self_deaf(true);  // <------ added

Is this the right place?

[Krzmbrzl]

No - you'll want to handle this on the client-side.

At first glance, I'd say somewhere here seems reasonable: https://github.com/mumble-voip/mumble/blob/master/src/mumble/MainWindow.cpp#L2790

[Hartmnt]

@Krzmbrzl I have trouble understanding which of OP's proposed features are supposed to be added. All 4 of them? Because, I actually think "(local) mute others while talking" could be a pretty good addition for people using voice activation or push-to-talk without headset.

@jaggzh I have worked on some parts of Mumble in the past which might be relevant. I hope I can give you some hints: I think you probably do not want to implement this in the MainWindow.cpp, but rather the mix() function in AudioOutput.cpp or the prepareSampleBuffer in the AudioOutputSpeech class. The first function mixes all voice and audio samples together, applies volume and also "priority speaker" status effects etc. The second function already takes all the different client audio states somewhat into account.

[Krzmbrzl]

Mute self when others are speaking

This is the option that I have first-hand experience with (Zoom uses/used that) and I found it worked quite nicely. However, since we'll want to implement this on a per-client basis, this requires client B to enable this setting for client A to not hear their echo. But then again, it's the same issue with general echo cancellation.

Change Audio in sensitivity when others are speaking

I don't see this being a viable option as the audio sensitivity is very device-specific and thus the amount by which it had to be increased to cut the echo does not seem to be obvious. Also, what to do for folks using continuous mode?

Deafen self when talking (push-to-talk-and-deafen)

This would be a good option but it has the problem that the echo comes back with some delay and that delay is not predictable. So in order to be sure to cut out echo, we'd have to keep the user deafened for a noticeable time after they have stopped speaking in which case we might cut off the beginning of an answer to what has been said, which I think will impair the conversation quality.

Mute others when talking (push-to-talk-and-mute-others)

By means of the end result, this seems to be identical to the option of deafening oneself while talking.

---

So all-in-all I think the option of muting oneself while others are speaking is probably the best solution :thinking:

[Hartmnt]

This is the option that I have first-hand experience with (Zoom uses/used that) and I found it worked quite nicely. However, since we'll want to implement this on a per-client basis, this requires client B to enable this setting for client A to not hear their echo. But then again, it's the same issue with general echo cancellation.

Interesting. I was under the impression that the client creating the echo would activate the "Mute self when others are speaking" to prevent echo... (Same for "local-mute-others-when-talking"). But maybe there are some drawbacks.

[Krzmbrzl]

Yes that's how it works, but the client creating the echo is not the one who hears the echo

[jaggzh]

I'd still need this for the use-case where the remote end is a webcam (picking up the whole room in continuous mode), for a patient to communicate with family, and the other computer needs to talk but gets feedback when they do. (The Plumble android client does the Push-to-talk in Walkie-talkie-mode thing, just fyi.)

I was thinking a separate option for a delay -- how long the self-deafen should continue after Push to Talk is released -- but for now I think it would work for most people to just hold it down longer. The user then has control. This would work for most cases, but wouldn't resolve if a speaker's in a noisy environment that continues to transmit the noise. Nevertheless, I think the basic option is a good first step.

[Krzmbrzl]

Noise != echo These are different issues that have to be handled differently.

I'd still need this for the use-case where the remote end is a webcam (picking up the whole room in continuous mode), for a patient to communicate with family, and the other computer needs to talk but gets feedback when they do.

I don't see why this would require special handling. The echo is suppressed if the receiver of audio data is muted while receiving audio.

One issue with this approach however is if someone is using continuous transmission mode, in which case the receiving end would be permanently be muted :thinking:

[jaggzh]

One issue with this approach however is if someone is using continuous transmission mode, in which case the receiving end would be permanently be muted thinking

Hence the non-automatic "Deafen self when push-to-talk" (ie. Walkie talkie mode)

[Krzmbrzl]

Fair enough :+1:

However, the implementation should also work with VAD, though I guess this should be easy enough.

So in the end, using the deafen approach might be the best option after all.

[jaggzh]

Just to not leave you hanging; I've been too busy and ended up making a walkie-talkie mode by using Push to Talk with a couple scripts assigned to keyboard keys. :/

mumble-talk-start:

#!/bin/bash
export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus
volmuteset 1
dbus-send --print-reply --type="method_call" \
        --session \
        --dest=net.sourceforge.mumble.mumble / \
        net.sourceforge.mumble.Mumble."startTalking"

mumble-talk-end

#!/bin/bash
export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus
dbus-send --print-reply --type="method_call" --session --dest=net.sourceforge.mumble.mumble / net.sourceforge.mumble.Mumble."stopTalking"
volmuteset 0