search

microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
909 stars 283 forks source link

Can't Deploy to Hololens #108

Closed javieros92 closed 4 years ago

javieros92 commented 4 years ago

Hello, I've been working on a project for Hololens with WebRTC using this library, previously I was capable of building and deploying it in the Hololens, but recently I've been unable to do it.

I'm not sure if the problem is the webRTC native dll or something else, but I have nowhere else to turn to for help.

Some of the things I've noticed is that when I Debug in the Emulator I get messages of not included with several PDBs, these are:

also I noticed it prompts me to load excptptr.cpp.

I'm using VS2019, Unity 2018.3.8f1 and it happens with both Hololens Emulator 1 and 2. Attached is a .txt with the output of Visual Studio. Debug.txt

The same application works no problem in Windows Standalone version. Any help would be super appreciated.

Edit: Forgot to add that I've noticed that it apparently fails to load the Native.dll because it expected a x86 architecture, but instead it's unknown, I'm not sure if that is the cause or not

djee-ms commented 4 years ago

Hi @javieros92,

Some of the things I've noticed is that when I Debug in the Emulator I get messages of not included with several PDBs

I don't think missing PDBs would prevent the emulator from running, although I never tried with an emulator, we generally do testing with actual devices.

it apparently fails to load the Native.dll because it expected a x86 architecture, but instead it's unknown

I assume you are talking about this:

Failed to load 'Microsoft.MixedReality.WebRTC.Native', expected x86 architecture, but was Unknown architecture. You must recompile your plugin for x86 architecture.

This is a Unity error which always appears, even when the plugins works fine, and has no impact. This is somewhat of a bug from Unity. I think they are trying to load the plugin as a native Unity plugin and failing because it's not, it's a native DLL but doesn't implement the Unity native plugin interface. The error message is confusing for sure, but you can safely ignore it.

Loaded 'U:\Users\DefaultAccount\AppData\Local\DevelopmentFiles\Microsoft.MixedReality.WebRTC.UnityVS.Debug_Win32.j_rom\Microsoft.MixedReality.WebRTC.Native.dll'. Symbols loaded.

The DLL is loaded, otherwise its PDB wouldn't be loaded.

(webrtcvoiceengine.cc:204): WebRtcVoiceEngine::WebRtcVoiceEngine

This is a log from WebRTC native, so the native DLL is necessarily loaded.

(impl_webrtc_audiodevicewasapi.cpp:1082): Using communications audio playout device: Handset (Null Audio Driver)

That "Null Audio Driver" sounds a bit fishy. Is there no audio output on the emulator?

avcore\audiocore\client\audioclient\audioclientcore.cpp(1404)\AUDIOSES.DLL!6BE7136F: (caller: 6BE719FC) ReturnHr(2) tid(15bc) 887C005E (impl_webrtc_audiodevicewasapi.cpp:697): Failed to configure input audio device after activate, hr=5454815c5453a93a-2004287478

This is most likely your actual error. The code is in impl_webrtc_AudioDeviceWasapi.cpp logs this message then throws a WinRT exception that eventually leads to some _CRT_ASSERT. Probably related to the Null Audio Driver above.

Network error trying to send data to http://127.0.0.1:3000/: Cannot connect to destination host

This also shows that the IP address of the node-dss server is incorrect, because I don't think you are running NodeJS on the HoloLens emulator. And without signaling you cannot establish a WebRTC connection. This is also a major error.

javieros92 commented 4 years ago

Hello @djee-ms ,

Thanks a lot for the answer. Regarding the IP address, I forgot to change it for that test, but when I used the proper address of the device where the node-dss server was running it was throwing some IL2CPP wrapper exceptions (I think was called like that, I can try to replicate that some time later).

Weirdly enough, it is now working. I am going to investigate further and try to find the cause of this problem, if I do I am going to comment it here.

Also I am going to check the Audio Driver issue, because despite the signaling now working I can't get Video or Audio to transmit, so I'll have to check further.

Anyway, thanks a lot for the help. I can't find the root of the issue but as I said I'll report it back if I do find it later.

JoelFrutiger commented 4 years ago

I had the same problem, you need to build the WSA/x86 version of the WebRTC.Native dll and set the ddl import settings accordingly, see https://microsoft.github.io/MixedReality-WebRTC/manual/helloworld-unity-importwebrtc.html.

To build the WSA/x86 version just set the CPU to x86 in the provided VS Solution and then rebuild the solution.

aleneum commented 4 years ago

I had a similar issue, in the emulator as well as on the actual device. The issue was quite simple to solve. I forgot to enable player capabilities such as WebCam, InternetClient, Microphone, SpatialPerception. I had to remove the previous build and do a fresh export. After that, the camera has been activated correctly and WinRT exceptions were no longer thrown.

djee-ms commented 4 years ago

@javieros92, @aleneum : Are things working for you now? Is there anything else related to this issue, or can I close it? I don't think we went to the bottom of your issue @javieros92 but I don't have enough info as is to identify a bug.

aleneum commented 4 years ago

@djee-ms: for me things work well 👍 , thanks!

javieros92 commented 4 years ago

@djee-ms I'm so sorry about that. We were in a big developing stage and I didn't have the time to study more the issue. At this point I think it had to do with some corrupted files because it hasn't happened again, in any of my current iterations of the project. So yes I think it would be ok to just close it. A bit disappointed I can't give you more info on the issue because it never appeared again.

(Now I do have another problem, but I don't know if I should open another issue or not. I have everything implemented and hosted on a server for the signaler. I can communicate between 2 devices, haven't tried with the hololens yet, if they are in the same network. But once I do it from different networks for each device the signaling occurs, the call starts but there's no video sent to the other side. I allowed both applications through the firewall to no avail and I can't pinpoint the issue)

djee-ms commented 4 years ago

@javieros92 : no worry, I just wanted to clean-up a bit the old issues. I am closing this one then.

For your other issue, no idea. It sounds like this is a firewall issue. Note that if you have 2 devices and 1 signaling server (so 3 machines in total) then you need both peers to communicate with the signaling server (for signaling), but also peer-to-peer with each other (for audio/video/data). It sounds like the former works, but not the latter. You can open a new issue if you feel this is a bug, but a priori it doesn't seem there is much we can do if that's related to networking/firewall.