Issue in Windows Mixed Reality

[awonnink]

Hi,

Dissonance doesn't seen completely UWP proof, or maybe better: Universal is not completely universal :)

Context

I have a more or less standard 'DissonanceSetup' in a player that works well op PC and HoloLens, using Dissonance 3.0.2.

Trying to build and run for Windows Mixed Reality however causes the player to stop at initialization. When I disable the DissonanceSetup gameobject (and skip the dissonance initialization) the project runs fine.

I will try to get more details, but maybe you already have hints about the possible cause or fix.

• Dissonance version used: v3.0.2
• Unity version: 2017.2.0f3-MRTP
• Editor Operating System and version: Microsoft Windows 10 Pro 10.0.10586 (x64), Windows falls update
• Build Settings:Windows UWP, x86_64, Development Build

ps. The problem is not related to the earlier issue with Math.DivRem: I have changed that part.

Albert Jan

[martindevans]

As far as I know you're the first person to use Dissonance on a mix reality headset! I had assumed that if it worked on Hololens it would just work^(tm) on the mixed reality headsets.

Could you up the logging level (Window > Dissonance > Diagnostic Settings) to Core:Trace and send me a log.

[awonnink]

I can't start the app directly from the Unity editor yet (need a real MR headset for that, because of a bug in Unity), or is this setting compiled into the player?

[martindevans]

Yep, those settings get built into Assets/Plugins/Dissonance/Resources/DebugSettings.asset and are loaded up at runtime.

[awonnink]

I narrowed the issue so far to that it happens when I call var comm = prefab.GetComponent(); Debug.Log("AJW: Starting InitializeAsClient:"); comm.InitializeAsClient(UrlHandler.SoundServer);

There is this 'Network access is denied', but apart from Dissonance the app also accesses the Network (using WWW), and that works fine. Maybe UWP has some extra restrictions on the ports allowed ...

output:

AJW: Starting InitializeAsClient:

(Filename: C:\buildslave\unity\build\artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)

The thread 0x11fbc has exited with code 0 (0x0). onecoreuap\analog\input\holographicdriverclientlib\lib\holographicdriverhandlewrapper.h(531)\Windows.Mirage.dll!13F7D611: (caller: 13F7D490) ReturnHr(4) tid(bd04) 80070041 Network access is denied. CallContext:[\OpenHolographicDevice] onecoreuap\analog\input\holographicdriverclientlib\lib\holographicdriverhandlewrapper.h(474)\Windows.Mirage.dll!13F7D46C: (caller: 13F7E241) Exception(2) tid(bd04) 80070041 Network access is denied. CallContext:[\OpenHolographicDevice] Exception thrown at 0x762408B2 in v-Slam Internet Browser.exe: Microsoft C++ exception: wil::ResultException at memory location 0x1439DEB8. Exception thrown at 0x762408B2 in v-Slam Internet Browser.exe: Microsoft C++ exception: [rethrow] at memory location 0x00000000. onecoreuap\analog\input\holographicdriverclientlib\lib\spatialgraphdriverclient.cpp(1009)\Windows.Mirage.dll!13F7E202: (caller: 13F7F152) ReturnHr(5) tid(bd04) 80070041 Network access is denied. Msg:[onecoreuap\analog\input\holographicdriverclientlib\lib\holographicdriverhandlewrapper.h(474)\Windows.Mirage.dll!13F7D46C: (caller: 13F7E241) Exception(2) tid(bd04) 80070041 Network access is denied. CallContext:[\OpenHolographicDevice] ] onecoreuap\analog\input\holographicdriverclientlib\lib\spatialgraphdriverclient.cpp(1441)\Windows.Mirage.dll!13F7F16E: (caller: 13F7EF9E) ReturnHr(6) tid(bd04) 80070041 Network access is denied. The program '[69568] v-Slam Internet Browser.exe' has exited with code 0 (0x0).

[martindevans]

Which network integration are you using? That definitely looks like some kind of permissions problem. I don't know much about permissions on these devices, I would have assumed they were the same for hololens and mixed reality :/

[awonnink]

I use the LL_API. A newer log also adds Log: cannot connect after {10} attempt address {::ffff:37.97.189.248:80} (changed the port to 80 just to check)

[martindevans]

Looking through the UWP permissions docs there are only two which may apply:

• Internet and public networks
• Homes and work networks

Perhaps the problem is you're accessing the internet (first permission) but if you're testing within the local network it needs the second permission?

[awonnink]

I included all the network related permissions, just to try (I also use WWW to het content from the same server, and that works). But without success. I have posted a question about the error also in the HoloDevelopers Slack group, maybe the can tell more (but it a bit quiet with the weekend starting :))

[awonnink]

I have found the main solution: apparently in UWP mixed reality I shouldn't do the Dissonance initialization at startup, but just after a few seconds. Furthermore, during my tests I switched between x86 and x64 versions, but the x86 version doesn't work completely correct on my system (possibly because the emulator is 64 bits, I don't know.) Anyway, I think I am good (I'll try the actual speaking tomorrow).

Thank you for your great assistance!

[martindevans]

Wow so just delaying the startup by a few seconds changes the permissions? That's pretty ugly! I'm glad you worked it out though :)

I'll close this issue now, feel free to re-open it if that test doesn't work tomorrow.

[awonnink]

Unfortunately I have to reopen this thread. The security errors disappeared when I started to test with x64 compilation again. The Dissonance related messages in the output are:

[Dissonance:Core] DissonanceComms: Loading default playback prefab

(Filename: C:\buildslave\unity\build\artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)

[Dissonance:Core] VoiceReceiptTrigger: Recalculating token activation: 0 tokens, activated: True

(Filename: C:\buildslave\unity\build\artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)

[Dissonance:Core] VoiceBroadcastTrigger: Recalculating token activation: 0 tokens, activated: True

(Filename: C:\buildslave\unity\build\artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)

..... .....

[Dissonance:Recording] CapturePipelineManager: Detected a frame skip of 154.3262ms, forcing reset of Microphone capture pipeline

Albert Jan

[martindevans]

Hi Albert,

What's the problem in that log you posted?

The first few messages are normal operation. The final message isn't ideal (it indicates that a single frame took 154ms) but when this happens it forces a reset which should ensure that voice works (with a small stutter caused by the reset).

[awonnink]

The log is not the problem. There are no errors showing, but the app just stops responding to input the moment I call InitializeAsClient. It might be that it somehow resets the UWP input devices, still testing ...

[martindevans]

Ah I see. Could you try finding this method in MicrophoneCapture.cs:

[CanBeNull] public static MicrophoneCapture Start([CanBeNull] string micName)

And make it always unconditionally return null. This will make Dissonance think there are no microphones available and will disable the capture system (no preprocessor, no codec for encoding), that'll cut out about half the possible source of error.

[awonnink]

I have traced the issue being caused by the the call of var clip = Microphone.Start(micName, true, 10, sampleRate); in Microphone manager. I guess this is something that should be fixed in Unity.

[awonnink]

Making Microphonecapture Start always return null let the app run correctly. I also tried using the first name from the Microphone devices in the Microphone.Start, but still the app freezes

[martindevans]

Yep that's definitely a Unity bug. I can't think of any good workaround either, the Microphone API is pretty simple so there's not much else to try :(

When you open a Unity issue could you post a link to it here? Thanks.

[awonnink]

I have created an almost empty scene with only one rotating Cylinder. After 10 seconds I call the UnityEngine.Microphone.Start() and the app freezes.

The unity issue: https://fogbugz.unity3d.com/default.asp?962086_g6vkojsl8pa0mj1b

[martindevans]

Thanks, hopefully Unity will get to fixing that soon.

If not then I might have a workaround (although it'll be fairly complex to use). Tom just merged in some work of mine for a "plug-in" microphone API which lets you replace the Dissonance Microphone recorder with your own mic recorder by implementing a certain interface. You could try using that to implement your own mic capture system directly using UWP audio APIs to bypass whatever the bug in Unity is. If you're interested in that send me an email and I can give you preview build (or can wait until the next version of Dissonance in a month or so).

[awonnink]

Last week I have received my mixed reality headset. It appears that the issue doesn't occur when the headset is attached. So, although the issue remains, it is not as inconvenient as I thought. (ps. the headset doesn't have a build-in microphone, it is still the same microphone I am using. The issue might be related with the App asking the user whether the use of the mic is allowed, which popup is not visible in the emulator)

[martindevans]

The linked Unity bug above has been closed, so I'll close this issue since I assume it's resolved.