[BUG]: Add Audio Track on Perfect Negotiation Cause Crashes in UnityEditor

[jeromegauzins]

Package version

3.0.0-pre.3

Environment

* OS: Windows 10
* Unity version:2020.3.23f1
* Unity Editor

Steps To Reproduce

• Implement a perfect WEBRTC negotiation between two separate computers on the same local network.
• Once the peers are connected, add an audiostreamtrack for the microphone on the peer on both computers and add receiver for this track on both computers too. (Same logic)
• This triggers a renegotiation and makes this crash appear quite often on Unity Editor.

Note:

• Adding a delay of 1 second when I add the audioStreamtrack on the sender side and also on the receiver side did not change anything.

Current Behavior

A Fatal Error Occured:

` #

Fatal error in: ..\src\modules\audio_coding\codecs\cng\audio_encoder_cng.cc, line 284

last system error: 0

Check failed: info.encoded_bytes == 0 (160 vs. 0)

Encoder delivered data too early.Crash!!!

SymInit: Symbol-SearchPath: 'C:/Program Files/Unity/Hub/Editor/2020.3.28f1/Editor/Data/Mono;.;C:\Users****;C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor;C:\WINDOWS;C:\WINDOWS\system32;SRVC:\websymbolshttp://msdl.microsoft.com/download/symbols;', symOptions: 534, UserName: '*' OS-Version: 10.0.0 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\Unity.exe:Unity.exe (00007FF749800000), size: 83247104 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 2020.3.28.16399 C:\WINDOWS\SYSTEM32\ntdll.dll:ntdll.dll (00007FFA0E330000), size: 2064384 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2130 C:\WINDOWS\System32\KERNEL32.DLL:KERNEL32.DLL (00007FFA0D250000), size: 782336 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2364 C:\WINDOWS\System32\KERNELBASE.dll:KERNELBASE.dll (00007FFA0BCB0000), size: 2957312 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2364 C:\WINDOWS\System32\CRYPT32.dll:CRYPT32.dll (00007FFA0C210000), size: 1400832 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2486 C:\WINDOWS\System32\ucrtbase.dll:ucrtbase.dll (00007FFA0C110000), size: 1048576 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.789 C:\WINDOWS\System32\USER32.dll:USER32.dll (00007FFA0D400000), size: 1708032 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2486 C:\WINDOWS\System32\win32u.dll:win32u.dll (00007FFA0BC80000), size: 139264 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2486 C:\WINDOWS\System32\GDI32.dll:GDI32.dll (00007FFA0D120000), size: 176128 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2130 C:\WINDOWS\System32\gdi32full.dll:gdi32full.dll (00007FFA0BB70000), size: 1110016 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2364 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\libfbxsdk.dll:libfbxsdk.dll (00007FF9860C0000), size: 10215424 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 2020.2.0.0 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\optix.6.0.0.dll:optix.6.0.0.dll (00007FF9FA550000), size: 208896 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 6.0.0.0 C:\WINDOWS\System32\msvcp_win.dll:msvcp_win.dll (00007FFA0C020000), size: 643072 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.789 C:\WINDOWS\System32\ADVAPI32.dll:ADVAPI32.dll (00007FFA0D010000), size: 712704 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2130 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\OpenImageDenoise.dll:OpenImageDenoise.dll (00007FF973AA0000), size: 43806720 (result: 0), SymType: '-deferred-', PDB: '' C:\WINDOWS\System32\msvcrt.dll:msvcrt.dll (00007FFA0D5B0000), size: 647168 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 7.0.19041.546 C:\WINDOWS\System32\sechost.dll:sechost.dll (00007FFA0D1B0000), size: 638976 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.1865 C:\WINDOWS\System32\RPCRT4.dll:RPCRT4.dll (00007FFA0C390000), size: 1200128 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2486 C:\WINDOWS\System32\SHELL32.dll:SHELL32.dll (00007FFA0C4D0000), size: 7618560 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2486 C:\WINDOWS\System32\SHLWAPI.dll:SHLWAPI.dll (00007FFA0D720000), size: 348160 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.2075 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\umbraoptimizer64.dll:umbraoptimizer64.dll (00007FF9C0760000), size: 1306624 (result: 0), SymType: '-deferred-', PDB: '' C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\FreeImage.dll:FreeImage.dll (0000000180000000), size: 6582272 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 3.18.0.0 C:\WINDOWS\System32\ole32.dll:ole32.dll (00007FFA0DCD0000), size: 1220608 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.1202 C:\Program Files\Unity\Hub\Editor\2020.3.28f1\Editor\ispc_texcomp.dll:ispc_texcomp.dll (00007FF9B6400000), size: 1609728 (result: 0), SymType: '-deferred-', PDB: '' C:\WINDOWS\System32\WS2_32.dll:WS2_32.dll (00007FFA0D6B0000), size: 438272 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.19041.546`

Expected Behavior

No Crash

Anything else?

Audio Codec used: audio/G722

[karasusan]

The line of this crash is here. https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_coding/codecs/cng/audio_encoder_cng.cc;l=284-285

This means there might be an issue about the size of audio data. How do you set audio data to AudioStreamTrack?

[jeromegauzins]

The line of this crash is here. https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/audio_coding/codecs/cng/audio_encoder_cng.cc;l=284-285

This means there might be an issue about the size of audio data. How do you set audio data to AudioStreamTrack?

Before I add the track, i initialize the microphone like this: ` foreach (var device in Microphone.devices) { Debug.Log("[MicroStreamer] Current Microphone Device = " + device);

            if (!microphoneBlackList.Exists(b => device.ToLower().Contains(b.ToLower())))
            {
                deviceName = device;
                break;
            }
        }

        Debug.Log("[MicroStreamer] Selected Microphone Device = " + deviceName);

        inputAudioSource.clip = Microphone.Start(deviceName, true, 2, frequency);
        // set the latency to “0” samples before the audio starts to play.
        int i = 0;
        while (!(Microphone.GetPosition(deviceName) > 0) && i < 100)
        {
            Thread.Sleep(10);
            i++;
        }

`

Here is the code where I add the track ` if (sendStream == null) sendStream = new MediaStream();

        if (audioStreamTrack == null)
        {
            audioStreamTrack = new AudioStreamTrack(inputAudioSource);
            audioStreamTrack.Loopback = false;
        }

        Debug.Log("[MicroStreamer] Add Track " + audioStreamTrack.Id);

        playerConnection.peer.AddTrack(audioStreamTrack, sendStream);

        var codecs = RTCRtpSender.GetCapabilities(TrackKind.Audio).codecs;

        List<RTCRtpCodecCapability> availableCodecs = new List<RTCRtpCodecCapability>();
        var excludeCodecTypes = new[] { "audio/CN", "audio/telephone-event" };
        foreach (var codec in codecs)
        {
            if (excludeCodecTypes.Count(type => codec.mimeType.Contains(type)) > 0)
                continue;
            Debug.Log("[MicroStreamer] Codec mimeType=" + codec.mimeType + " sdpFmtpLine=" + codec.sdpFmtpLine + " clockRate=" + codec.clockRate + " channels=" + codec.channels);
            availableCodecs.Add(codec);
        }

        RTCRtpTransceiver transceiver = playerConnection.peer.GetTransceivers().First();
        RTCErrorType error = transceiver.SetCodecPreferences(availableCodecs.FindAll(c => c.mimeType.Contains("audio/G722")).ToArray());
        if (error != RTCErrorType.None)
            Debug.LogError(error);`

[karasusan]

@jeromegauzins Thank you for sharing snippets. Can you make the simple Unity project to reproduce the crash?

[karasusan]

@jeromegauzins I would like to reproduce the issue in my environment. Can you help me to do it?

[jeromegauzins]

I'm afraid I can't give you source code like that.

To reproduce this problem:

• Have two computers running Windows 10 - Unity Editor 2020.3.28f1 on the same local network
• Have a negotiation that open datachannel (polite system), via a perfect WEBRTC negotiation.

On each computer once the negotiation is terminated and the peer connected:

• Send a ping message and wait for confirmation from the other
• Once this ping is validated, add an audio track like the code snippet I showed you before.
• Exit the lobby by destroying the peer and enter again: recreating the peer, open the data channel, send ping, and add track several times.
• Normally it should appear quite often

More info abour my project:

• Signaling via Socket.Io
• 2 datachannels are created when the peer are created
• ILCPP (Arm64)
• .NET 4x

I've got theses stuns and a TURN:

When the peer is created, I add a Transceiver RecvOnly

Note:

• It never seems to crash in Android build, but only in Unity Editor from time to time.
• Only the polite peer create the datachannels, but both do the ping and the VOIP setup.

Feel free to ask me more questions

[karasusan]

@jeromegauzins We are checking the information you shared, but it is difficult to reproduce the crash. I'm glad if we have the minimum project to reproduce it.