[bug] Voice chat randomly stops working, more likely with more players.

Context

In our game voice chat will just randomly break (no one will hear eachother), sometimes it will come good, sometimes it will stay broken until the round restarts and scene changes. This is more common if youre a later joining player and with more than 20 players on a server. It doesnt affect all players usually but it will affect a good portion at the same time.

Ive noticed a few exceptions/errors in our logs:

This one seems to be the handshake failing? this one gets spammed a lot at times.

IndexOutOfRangeException: Array index is out of range.
  at Dissonance.Networking.PacketWriter.Write (Byte b) [0x00000] in <filename unknown>:0 
  at Dissonance.Networking.PacketWriter.Write (UInt16 u) [0x00000] in <filename unknown>:0 
  at Dissonance.Networking.PacketWriter.WriteHandshakeResponse[HlapiConn] (UInt32 session, UInt16 clientId, System.Collections.Generic.List`1 clients, System.Collections.Generic.Dictionary`2 peersByRoom) [0x00000] in <filename unknown>:0 
  at Dissonance.Networking.Server.BroadcastingClientCollection`1[Dissonance.Integrations.UNet_HLAPI.HlapiConn].ProcessHandshakeRequest (HlapiConn source, Dissonance.Networking.PacketReader& reader) [0x00000] in <filename unknown>:0 
  at Dissonance.Networking.BaseServer`3[Dissonance.Integrations.UNet_HLAPI.HlapiServer,Dissonance.Integrations.UNet_HLAPI.HlapiClient,Dissonance.Integrations.UNet_HLAPI.HlapiConn].NetworkReceivedPacket (HlapiConn source, ArraySegment`1 data) [0x00000] in <filename unknown>:0 
  at Dissonance.Integrations.UNet_HLAPI.HlapiServer.OnMessageReceivedHandler (UnityEngine.Networking.NetworkMessage netmsg) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkConnection.HandleReader (UnityEngine.Networking.NetworkReader reader, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkConnection.HandleBytes (System.Byte[] buffer, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkConnection.TransportReceive (System.Byte[] bytes, Int32 numBytes, Int32 channelId) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServer.OnData (UnityEngine.Networking.NetworkConnection conn, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServer+ServerSimpleWrapper.OnData (UnityEngine.Networking.NetworkConnection conn, Int32 receivedSize, Int32 channelId) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServerSimple.HandleData (Int32 connectionId, Int32 channelId, Int32 receivedSize, Byte error) [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServerSimple.Update () [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServer.InternalUpdate () [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkServer.Update () [0x00000] in <filename unknown>:0 
  at UnityEngine.Networking.NetworkIdentity.UNetStaticUpdate () [0x00000] in <filename unknown>:0 
(Filename:  Line: -1)

We also have this spammed quite a bit:

[Dissonance:Network] (15:03:24.135) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.193) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.193) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.193) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.193) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.193) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.226) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.226) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.253) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.659) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.721) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.752) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.
[Dissonance:Network] (15:03:24.970) HlapiServer: Received a packet with incorrect session ID. Expected 140609981, got 205130252. Resetting client.

Expected Behavior

Voice chat should be useable throughout the game

Actual Behavior

Voice chat will stop working at a point in the round, usually will be good next round and then break again.

Steps to Reproduce

Cant :(, We cant reproduce this locally, its happening on game servers hosted by our community.

Your Environment

Dissonance Version: v6.0.2

I think you may be hitting a limit in the maximum size of packets which Dissonance can send (1024 bytes). The handshake packet is by far the largest packet because it sends a list of everyone who is in the session - all of those large string overflow the packet size. I should probably increase the maximum packet size.

Until I've done that search within Assets/Plugins/Dissonance/Core/Networking for 1024 and increase all the packet sizes to 4096 - that should handle up to 80 players. You will also need to do the same thing to the HLAPI integration.

Gave it a test today with +20 players, didnt see that exception but issue still seemed to happen. This time "no free events for message in the pool", might not be a dissonance issue and instead too much network traffic on our side.

no free events for message in the pool

That's a HLAPI message, Check out this thread for more details. It seems to indicate one of two things:

One possibility is that the send buffer is full, i.e. simply too much network traffic. This seems unlikely, since you presumably didn't have this issue before now (and this change didn't introduce any more packets, just larger ones).

The other possibility is that the message is too large for HLAPI to send in one piece, confusingly it seems HLAPI uses the same error message for this situation. If you're using Unity 2018.1 you may be able to use QosType.ReliableFragmentedSequenced instead of ReliableSequenced for the Dissonance channel, I haven't tried this myself but it looks like it should work.

I did some work last night to properly solve this problem so that Dissonance will handle arbitrarily large session sizes - essentially changing it so that Dissonance will fragment the handshake packet itself. Unfortunately I'm away for the Unity Unite Berlin conference next week, so I won't be able to properly test that until afterwards - sorry about that :(

I'm back from the conference. I've just added some finishing touches to a solution to this and tested it with a smaller number of local clients.

The problem with the old system was that the complete session state is packed into a single "handshake response" packet, with 20 players this packet gets too large. The new system sends back a handshake response indicating that the session you've just joined is empty and then immediately follows it up with a packet per remote player, as if they've all joined the session. This should keep the size of each individual change and as a bonus is also backwards compatible with older servers and clients!

Could you drop me an email (martin@placeholder-software.co.uk) with your invoice number, I'll send you back a copy of Dissonance with these changes built in so that you can test it. @GabeBigBoxVR you may also want to do the same.

A fix for this has been released onto the asset store in Dissonance 6.2.3.

This involved a change to the network handshake system. In the old system a single HandshakeResponse packet was sent from the server to a new client, this packet contained the complete state of the session (all other clients, their names, their codec settings, list of rooms they're in) - obviously this packet eventually would grow too large and overflow the 1024 byte limit. In the new system the HandshakeResponse packet comes back with no data in it (as if the session is empty) and is then followed by one packet per client which contains the information about that client (as if they are all just now joining) - this should ensure that no single packet ever grows too large no matter how big the session gets. This change is completely backwards and forwards compatible with other versions of Dissonance.

Placeholder-Software / Dissonance