"Echo" problem that scales with number of players (LAN)

[kristianpd]

Context

I'm encountering an issue with a relatively straightforward setup. A global room (single root level game object with Broadcast and Receipt triggers on Global Room) setup with 4 or more players begins to echo and scales with the number of players. Our facility has 6 seats, spread out over two separate rooms (4 and 2). When testing multiplayer at home, there is no echo that occurs when running multiple clients off the same machine. When testing multiplayer at our facility, the echo seems to scale with the number of players.

I should point out, I don't believe this is necessarily a real echo, in that I am not sure it's related to the voice actually being picked up. It's my best guess that there seems to be some sort of post processing that results in the echo of the player hearing their own voice as if it was being transmitted through their mic (not being picked up on other mics).

I think the problem here is some form of processing behaviour, that is magnified when more latency and more broadcasters are introduced.

Expected Behavior

We are currently running an old (1.5yrs +) version of Dissonance without this issue. This is my second attempt at updating this library but both have failed for similar reasons. I do expect a little echo, a single echo, where I would hear my own voice in the background on other player mics. I know there are some options for trying to eliminate this but the background noise of my own voice isn't the problem.

Actual Behavior

I apologize if this is a vague, but the actual behaviour is best described as follows:

• Multiple local clients on same machine have no issue (i suspect this is related to no latency, or sharing the same mic)
• Running 2 / 6 players in our facility does not seem to have this issue. The more players I add (4 to 6), begins displaying some kind of echo that scales (number of echos) with the number of players connected. The weird behaviour here is that the echo is not necessarily the other mics picking up my voice (that's expected and is a little present) but rather that my own voice seems to play back at full, then diminishing volumes (each echo diminishes), based on the number of players.
• I have tested this a bunch by muting mics, covering my voice, talking quietly, etc, and it definitely seems related to the input on my own device rather than other devices picking it up. I thought maybe it could be some post processing that is occurring where the sound profile of my own voice sounds slightly similar to the sound profile of the faint echo in background, but I feel like I have done a decent job at eliminating the chance that other mics are picking me up.
• Player 1-6 are connected. Player 1 talks in mic (mic pressed up against lips so loud locally but not loud to other mics). The sound would be something like me saying: "Testing" and after a short delay, on a consistent interval I would hear: "TESTING" "TESting" "testing" "test..." (sound diminishes). On a 4 player setup it may be "TESTing" "Testing" "te..." (fewer "echos").

I've had a real hard time narrowing this. I've tried eliminating all other forms of windows/mic configuration issues and at this point I'm really thinking it's a Dissonance thing. It's also only present in the last few version of dissonance (I tried updating to v4 and then v6) and not so in an old (1.5yrs) version. In the old version, the expected behaviour of hearing my faint voice in the background after a slight latency delay (just once) is what I would expect in this newer version.

Workaround

I cannot find a workaround for this, nor can I seem to reproduce it locally.

I have tried playing with various voice settings including: Noise supression, and Acoustic echo cancellation. I have also disabled filters in Advanced desktop options to no avail (if anything it may have been slightly worse?)

Steps to Reproduce

I have been able to reproduce this (in the 4.0 version, but identical problems) on a simple project

1. Set up a basic project with root game object DissonanceSetup and another root object with a Global room broadcast/receipt trigger
2. Build the project and deploy to 4 or 6 machines (gets worse with more)
3. Place 4 machines in same room (I am not convinced this is the problem, but this is our setup)
4. Start talking or having you and others talk at the same time. The problem seems to be magnified by volume of players speaking (into their own mics) (can't 100% confirm this)

Your Environment

• Dissonance version used: 6.2

• Unity version: 2017.4.11

• Editor Operating System and version: Windows 10

• Build Settings: Regular windows build

I'll try to answer any questions, hope this gives a starting point.

[martindevans]

This is an odd one, no one else has reported hearing their own voice and such a thing is basically impossible with Dissonance because there is no audio pipeline allocated for your own voice, one being created and properly working entirely by accident is pretty unlikely! There's also no mixing of voices performed by Dissonance at any stage of it's pipeline, so it can't be your voice getting mixed into a signal with someone else's voice on the server side. I can't think of any mechanism for your own voice to be played through your own speakers except a remote player somehow transmitting it back to you, I think this is backed up by the fact that it scales with the number of players - they're likely each individually sending back your voice.

So I think there are two questions to answer:

• Is this a bug in Dissonance (causing remote players to send back your voice somehow), or is it a simple mic echo?
• Why doesn't this happen on the older version?

My suspicion is that it's mic echo, to confirm that if it's not can you:

• Mute everyone (except yourself) in the session (or use push-to-talk) (to ensure they can't be transmitting it back)
• Deafen everyone (except yourself) in the session (to ensure that your mic can't pick up their speakers/headphones playing your voice)

To answer the second question there are two changes which I think may be pertinent:

• In Dissonance 1.0.8 we replaced the voice detector with one based on the webrtc voice detector. This is an excellent voice detector which is much more sensitive than the old one - i.e. it's possible that the extra sensitivity is picking up the voices it didn't before (assuming you're using voice activation?).
• In Dissonance 2.0.0 we replaced the old speex preprocessing pipeline with the webrtc preprocessing pipeline. The new pipeline has much better adaptive gain control - if it detects a voice in the mic signal (no matter how quiet) it will boost it up to an audible level (it's really quite amazing how much it can boost a voice). So it's possible that the echoes existed before but were inaudible, and the new AGC is boosting them up.

An extra question: Have you used the acoustic echo cancellation system? It won't work well in a closed environment with many voices overlapping, most likely that wouldn't cause any issues but it's worth disabling it just to be sure.

[kristianpd]

I don't believe it's a simple mic echo. Another theory was that perhaps the new and improved voice processing may be picking up similar sounds and therefore amplifying them?

The part I can't explain is why speaking into a mic (mouth against mic, hands cupped around to try and reduce any sound. speaking or blowing into mic) results in a louder echo in my own headphones. When I say echo I don't think it's actually the other mics picking it up, but basically a local representation of my own voice that comes back after a latency period of some sort, then repeats on that interval until it fades.

I could understand a louder echo being picked up on other mics when talking loudly, but hearing it locally in a louder volume when I blow into my mic for example doesn't make any sense to me :.

I have tried (in the past) muting the mic's and still had a similar issue.

I did try the acoustic echo cancellation in a quick build i made to try and eliminate this issue (along with turning off the two filters in advanced options), and it didn't make a difference (if anything it may have been slightly louder with all those options changed).

It's very possible that the AGC is causing an issue for me that wasn't there before, but the echo just doesn't sound right to me. I apologize for over-using the word echo here. Before, as expected, i would hear a single echo as the other mics picked up my voice and played it back. Even if that was louder now (very likely), the multiple echo (more echos more players) doesn't really make sense.

I've written you an email at placeholder software to give you my personal email, if we could do a quick hangout / discord chat I think I can much better explain the behaviour!

[martindevans]

Discussed further on discord, it sounds like it's probably remote mics echoing back audio although that's not entirely certain.

A few ideas for further testing:

• Mute all players (except yourself)
• Deafen all players (except yourself)
• Lower Audio Duck Attenuation a lot (should reduce the volume of echoes)
• Attenuate the mic signal a lot (should force echoed voices to be below the level where the VAD/AGC can detect them)
• Print out the IsSpeaking flag for all players, to verify if they are speaking when an echo comes back

[martindevans]

Since we never really established what the problem was here (besides a lot of echoes from the environment) I'm going to close this issue. Feel free to re-open it or to open a new one if the problem persists :)