[bug] Plugin for Magic Leap missing Dissonance_CreateRnnoiseState

[nieman1234]

When I mark Assets\Plugins\Dissonance\Plugins\Magic Leap\libAudioPluginDissonance.so to be included in Magic Leap builds I get an error that I do not get when including Assets/Plugins/Dissonance/Plugins/Android/libs/ARM64/libAudioPluginDissonance.so instead.

Documentation states to include Assets\Plugins\Dissonance\Plugins\Magic Leap\libAudioPluginDissonance.so for Magic Leap builds.

When include the plugin from Assets\Plugins\Dissonance\Plugins\Magic Leap\libAudioPluginDissonance.so I get this error:

2021-08-13 13:52:00.372;116008;976;1;Info;Unity;DEBUG [Dissonance:Recording] (17:52:00.239) WebRtcPreprocessingPipeline: Resetting RnnoisePreprocessor;
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object);
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]);
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;UnityEngine.Logger:Log(LogType, Object);
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;UnityEngine.Debug:Log(Object);
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;Dissonance.LogMessage:Log();
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;Dissonance.Logs:WriteMultithreadedLogs();
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;Dissonance.DissonanceComms:Update();
2021-08-13 13:52:00.372;116008;976;1;Info;Unity;;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;[Dissonance:Recording] (17:52:00.251) BasePreprocessingPipeline: Voice Error: Unhandled exception killed audio preprocessor thread: System.EntryPointNotFoundException: Unable to find an entry point named 'Dissonance_CreateRnnoiseState' in 'AudioPluginDissonance'.;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at Dissonance.Audio.Capture.AudioPluginDissonanceNative.Dissonance_CreateRnnoiseState () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at Dissonance.Audio.Capture.WebRtcPreprocessingPipeline+RnnoisePreprocessor.Reset () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at Dissonance.Audio.Capture.WebRtcPreprocessingPipeline.ApplyReset () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at Dissonance.Audio.Capture.BasePreprocessingPipeline.ThreadEntry () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Action.Invoke () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ContextCallback.Invoke (System.Object state) [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ThreadHelper.ThreadStart () [0x00000] in <00000000000000000000000000000000>:0;
2021-08-13 13:52:00.380;116008;976;1;Error;Unity;at System.Threading.ThreadStart.Invoke () [0x00000] in <00000000000000000000000000000000>:0 ! Error ID: 02EB75C0-1E12-4109-BFD2-64645C14BD5F;

Then the preprocessor buffer fills and app performance tanks.

2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;[Dissonance:Recording] (17:52:00.935) BasePreprocessingPipeline: Lost 960 samples in the preprocessor (buffer full), injecting silence to compensate;
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[]);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;UnityEngine.Logger:Log(LogType, Object);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;UnityEngine.Debug:LogWarning(Object);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.LogMessage:Log();
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Logs:SendLogMessage(String, LogLevel);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Log:WriteLog(LogLevel, String);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Log:WriteLogFormat(LogLevel, String, TA);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Log:Warn(String, TA);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.BasePreprocessingPipeline:Dissonance.Audio.Capture.IMicrophoneSubscriber.ReceiveMicrophoneData(ArraySegment`1, WaveFormat);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.BasicMicrophoneCapture:SendFrame();
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.BasicMicrophoneCapture:ConsumeSamples(ArraySegment`1);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.BasicMicrophoneCapture:DrainMicSamples();
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.BasicMicrophoneCapture:UpdateSubscribers();
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.Audio.Capture.CapturePipelineManager:Update(Boolean, Single);
2021-08-13 13:52:00.958;116008;976;1;Warn;Unity;Dissonance.DissonanceComms:Update();

I got around this issue by including Assets/Plugins/Dissonance/Plugins/Android/libs/ARM64/libAudioPluginDissonance.so instead of Assets\Plugins\Dissonance\Plugins\Magic Leap\libAudioPluginDissonance.so Using this work around Dissonance functions and more than one Magic Leap headset can communicate with each other via Dissonance.

• Dissonance v8.0.1
• Unity 2020.3.8f1
• Editor OS: MacOS BigSur 11.5.1 (20G80)

[martindevans]

This is a mistake on my part when setting up the AudioPluginDissonanceNative class - the new RNN noise removal system isn't tested on the magic leap and isn't intended to be used on that platform (the ML has quite a lot of it's own mic preprocessing, so the extra cost of RNNoise on top of that probably isn't worth it).

In Assets/Plugins/Dissonance/Core/Audio/Capture/AudioPluginDissonanceNative.cs on line 27 find this line:

#if UNITY_EDITOR_WIN || (UNITY_STANDALONE_WIN && !UNITY_WSA) || UNITY_EDITOR_LINUX || UNITY_STANDALONE_LINUX || UNITY_ANDROID || UNITY_LUMIN

and remove UNITY_LUMIN from the end. This will disable the RNNoise removal system for Lumin and prevent the error form happening.

[nieman1234]

Thank you so much I was able to test this during the week, but I'm still having issues with the Magic Leap version of the plugin.

On line 110 of AudioPluginDissonanceNative.cs I had to add UNITY_LUMIN when using the plugin from Assets/Plugins/Dissonance/Plugins/Android/libs/ARM64/libAudioPluginDissonance.so

Seems the Magic Leap plugin doesn't include Dissonance_SetAgcIsOutputMutedState and when I leave UNITY_LUMIN out of the platform directive on line 110 I get a compiler error with 'AudioPluginDissonanceNative.Dissonance_SetAgcIsOutputMutedState(IntPtr, bool)' is inaccessible due to its protection level from Assets/Plugins/Dissonance/Core/Audio/Capture/WebRtcPreprocessingPipeline.cs(220,49)

I'm not sure what gives, but I do have a ML One for testing.

I'm able to get around this by using the Assets/Plugins/Dissonance/Plugins/Android/libs/ARM64/libAudioPluginDissonance.so version of the plugin and adding UNITY_LUMIN to line 110 in Assets/Plugins/Dissonance/Core/Audio/Capture/AudioPluginDissonanceNative.cs

[martindevans]

Ah sorry, the Dissonance_SetAgcIsOutputMutedState should be public (another mistake I made while setting up the AudioPluginDissonanceNative class) - that's been fixed in the next version of Dissonance (probably coming next week).

[martindevans]

Using the Android version of the plugin is also ok.

We kept the two separate because the ML does quite a lot of audio preprocessing before Dissonance has access to the audio, so it seemed likely that we might want to have a different audio processing system for ML audio. However, at the moment they're pretty much the same in the C++ code with just some differences in the C# code configuring it.

[nieman1234]

Thank you so much! I'm just going to keep running my workaround setup until that version releases. Loving the plugin btw!

[martindevans]

Dissonance 8.0.2 was release a while ago which should fix this issue.