Mixed Reality Capture for HoloLens

[mluketin]

Hi, thank you for examples.

I made a call between laptop and hololens and noticed that hololens sends plain video without holograms. Will you enable mixed reality capture on hololens?

[alvaroZerintia]

Hi mluketin, could you do a videoCall on Hololens? What sample did you run? I tried to run AppRTC-UWP sample on Hololens but doesn't work, however it works fine on Desktop App.

[mluketin]

I used ChatterBox-Sample. Server on laptop, one client on laptop, another client on hololens.

[alvaroZerintia]

Right, with ChatterBox-Sample is only audio calls? Or you could send video from Hololens?

Thanks in advance

[mluketin]

You can make video calls. But video from hololens does not contain holograms, only plain video.

[djungalio]

Is there any light on this topic? I am interested in video chat with holograms

[adrianimajion]

Has anyone had performance problems with the hololens streaming? I can see that it outputs at 30-32fps but the client receives it (same local network) at 2fps.

[jamescadd]

@mluketin While we don't yet have a sample for mixed reality capture, the documents for implementing this yourself are available here: https://developer.microsoft.com/en-us/windows/mixed-reality/mixed_reality_capture_for_developers I would recommend starting with the PeerCC Sample rather than Chatterbox as it is much simpler.

@adrianimajion I believe we've already caught up on email, but the latest PeerCC sample has vastly improved performance over the Chatterbox sample. Please open an issue if you see any performance issues.

Thanks, James

[gitfrida]

@jamescadd do you know what is the frame rate improvement using the highest camera resolution? also can you tell which percentage of the CPU is used? would we be able to display some AR while streaming?

[jamescadd]

@gitfrida I tested 640x480 coming from remote peer, 720p from HoloLens camera and saw 80%cpu and 100% gpu utilization. it's definitely not hitting our targets yet, but you could try with AR and see what mileage you get. previously we were having trouble supporting basic bi-directional streaming which is what this update focused on. We do have plans for an improved UWP rendering engine which should give you the ability to add AR, currently estimate that will be done in about 3-4 weeks. I'll post the update here when it's ready or feel free to ping me at jacadd@microsoft.com for progress.

[jamescadd]

Update to note that the following API will be supported as well: https://docs.microsoft.com/en-us/uwp/api/Windows.Media.Capture.MediaCapture#Windows_Media_Capture_MediaCapture_AddVideoEffectAsync_Windows_Media_Effects_IVideoEffectDefinition_Windows_Media_Capture_MediaStreamType_

[ghost]

Hello guys, any update on this topic?

[jamescadd]

Hi all, our latest branch is below. The performance issues are solved and MRC work is currently underway. Right now we're seeing a conflict between MRC and WASAPI that we're working to resolve, and there's also a bug that manifests when both ends of the call are Unity clients. This is much closer to being resolved but still not ready for production:

Prerequisites: Visual Studio 2017

• Windows SDK 10.0.14393
• Visual Studio Tools for Unity
• Visual Studio Tools for Universal Windows Apps Perl for Windows - http://strawberryperl.com/

1) Install Unity Editor 2017.2.0f3 with Metro Support https://download.unity3d.com/download_unity/46dda1414e51/Windows64EditorInstaller/UnitySetup64-2017.2.0f3.exe http://download.unity3d.com/download_unity/46dda1414e51/TargetSupportInstaller/UnitySetup-Metro-Support-for-Editor-2017.2.0f3.exe 2) Clone the repository: git clone --recursive -b Mosa/20180223-unity-hololens https://github.com/ortclib/ortclib-sdk.git ortclib-sdk-m62-media-engine-video 3) From a command prompt run bin\prepare.bat from the ortclib-sdk-m62-media-engine-video directory 4) Open ortclib-sdk-m62-media-engine-video\webrtc\windows\solutions\WebRtcUnity.sln 5) Select Debug configuration and x86 platform (others can be used as well) 6) Build project PeerConnectionClientUnity.WebRtc

[jamescadd]

Latest bits are below - tested and working on PC but pending testing on HoloLens. MRC code is in place but not yet tested, perf hasn't been verified either.

https://github.com/ortclib/ortclib-sdk/tree/Mosa/20180320-peercc-unity

The desktop app should be the standard PeerConnection M62 build Codec is forced to H.264 and settings are hard coded. Peer selection list isn't functional Enter the signaling server IP into the text box (HoloLens keyboard isn't implemented yet so use the virtual input feature on the device portal) Click the connect button in the Unity app, peer should show up on desktop Click the call button on Unity app (this app must initiate for now)

[ghost]

We're connecting UWP clients (including HoloLens running a Unity3D app) to a WebRTC video bridge, and until yesterday it was working fine. We're using XMPP Jingles for signalling. After an overnight update from Microsoft, however, the HoloLens client has stopped working. The Desktops are still OK. I built the WebRtcUnity project as above, and debugged the HoloLens client with that. I'm seeing an access violation when I set the local description, after having received the remote SDPf rom the session-initiate Jingle. I'm taking care to run the RTCPeerConnection methods on the same (CoreDispatcher) thread. The pertinent log output is as follows:

`(webrtcsession.cc:692): Local and Remote descriptions must be applied to get the SSL Role of the session. (webrtcsession.cc:692): Local and Remote descriptions must be applied to get the SSL Role of the session. (webrtcsession.cc:692): Local and Remote descriptions must be applied to get the SSL Role of the session. (messagequeue.cc:534): Message took 64ms to dispatch. Posted from: webrtc::PeerConnectionProxyWithInternal::CreateAnswer@C:\Users\Colin\Source\Hololens\Ortc\ortclib-sdk-m62-media-engine-video\webrtc\xplatform\webrtc\webrtc/api/peerconnectionproxy.h:73 The thread 0x74 has exited with code 0 (0x0). (webrtcvideoengine.cc:1217): Call stats: 1523986741791, {send_bw_bps: 800000, recv_bw_bps: 0, max_pad_bps: 0, pacer_delay_ms: 0, rtt_ms: -1} (transportcontroller.cc:610): Set local transport description on audio (p2ptransportchannel.cc:326): Set ICE ufrag: Y/fM pwd: CDsRQ0sZDyP86fLGUWgf7gtk on transport audio (dtlstransport.cc:317): Jingle:DtlsTransport[audio|1|]: DTLS setup complete. (transportcontroller.cc:610): Set local transport description on video (p2ptransportchannel.cc:326): Set ICE ufrag: Y/fM pwd: CDsRQ0sZDyP86fLGUWgf7gtk on transport video (dtlstransport.cc:317): Jingle:DtlsTransport[video|1|]: DTLS setup complete. (transportcontroller.cc:610): Set local transport description on data (p2ptransportchannel.cc:326): Set ICE ufrag: Y/fM pwd: CDsRQ0sZDyP86fLGUWgf7gtk on transport data (dtlstransport.cc:317): Jingle:DtlsTransport[data|1|__]: DTLS setup complete. (channel.cc:864): Channel enabled (channel.cc:1756): Changing voice state, recv=0 send=0 (channel.cc:864): Channel enabled (channel.cc:2018): Changing video state, send=0 (webrtcsession.cc:863): Session:3714235200520146857 Old state:STATE_RECEIVEDOFFER New state:STATE_SENTPRANSWER (channel.cc:1769): Setting local voice description (webrtcvoiceengine.cc:1397): WebRtcVoiceMediaChannel::SetRecvParameters: {codecs: [AudioCodec[0:PCMU:8000:0:1]], extensions: [{uri: urn:ietf:params:rtp-hdrext:ssrc-audio-level, id: 1}]} (webrtcvoiceengine.cc:1566): Setting receive voice codecs. (webrtcvoiceengine.cc:1854): AddSendStream: {id:audio_label_669db718a9eb8bee;ssrcs:[241845736];ssrc_groups:;cname:0kw+ABbwRzLpGweM;sync_label:stream_label_82ec0c960d4c6ff} Exception thrown: read access violation.

`I'm happy to send code and SDPs if it helps, and test anything that might help. Setup: HoloLens- Windows version 14393.2189.x86fre.rs1_release.180329-1711 Unity - 2017.3.1f1 Visual Studio 2017 Windows SDK 10.0.16299

[jamescadd]

@dixoncb Looking into the above. small update on the unity solution - its functionally pretty close to complete. The resolution can be selected in ControlScript.cs in the Assembly-Csharp project which builds out of VS. 2 way calling works but we're not seeing great performance. The issue has been tracked down to colorspace conversion in libyuv and our next steps will be to remove the dependency. On the MRC front we're still investigating why the current code isn't working and I'll update the thread when the issue/resolution is known. MRC is enabled in video_capture_uwp.cc line 420 for the curious.

[jamescadd]

@dixoncb It looks like a repro would be helpful to fix the issue, tough to diagnose with that output. I did notice you're on the current production HoloLens OS, if you can try it on Insider it would be interesting to see if it still happens. If there's a way to repro this with the Unity PeerCC app let me know, or if you're willing to share code feel free to contact me at jacadd@microsoft.com and we can start there.

[vgibilmanno]

Hi, great work so far! MRC works for me on the hololens with the peercc unity branch @jamescadd posted above. I enabled HologramCompositionEnabled in the video_capture_winuwp.cc file to make sure, that it is enabled but I think the default is already true.

But even though MRC works, I can confirm the huge performance issue. Video and voice are delayed by half a minute or even more.

EDIT: My bad. After building everything as release instead of debug, the performance is actually very good! I compared two scenarios. One with the libyuv dependency removed and one without removing libyuv, but performance wise nothing seemed to have changed.

The only thing I noticed now is that the 3D objects don't stay on their fixed position. They wander around especially when I slowly move my head. I linked a gif showing what I mean.

GIF here

[jamescadd]

@vgibilmanno Great to hear on both fronts! I wonder if some of the wandering is related to the known issue here: https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality-capture-for-developers Are you able to comment? If you can share a sample of the drifting we can take a look to see if anything obvious is wrong. cc @morosev

[jamescadd]

Update regarding MRC - as others have pointed out, it is in fact working for app-level holograms :) It does not however work for system level holograms, so if you use the Holograms app to place one it will not get picked up by the in-app MRC. This is by design on the HoloLens side and we cannot work around the limitation at this time.

The next steps will be to merge this into the master PeerCC sample and update the NuGet package to make things easier to build, when that's complete we'll close the issue. I'm also aware that there are other outstanding items for HoloLens such as access to the raw media objects and spatial audio, but those are reflected in separate tickets. Appreciate everyone's feedback and help testing so far!

Latest code is built against Unity 2017.4 and available in this branch:

git clone --recursive -b Mosa/20180320-peercc-unity https://github.com/ortclib/ortclib-sdk.git ortclib-sdk-m62-media-engine-video

[vgibilmanno]

Yes the known issues and the provided solutions helped me fix the problem. I had to either better map my room (I was just looking to a wall the whole time) or introduce a focus point with a plane. In fact the MRC on the developer portal had the same drifting effect before applying those solutions.

I would say the drifting thing has nothing to do with this issue. So yeah everything seems to work :)

[jamescadd]

@vgibilmanno I wanted to follow up on your comment about not using libyuv - did you replace the colospace conversion with a shader or something similar? We want to get this workload running on GPU and test performance, wondering if you've already done this. Thanks!

[vgibilmanno]

@jamescadd I removed the reference to libyuv and the usages without replacing it with anything.

[DanAndersen]

Hi, I am looking to use the Mosa/20180320-peercc-unity branch and seeing that @vgibilmanno has successfully gotten MRC with WebRTC from HoloLens to desktop is very encouraging. I was curious: did you have to upgrade your HoloLens to the RS4 preview or were you able to hold off? I notice that the VS2017 projects in all the code are requiring a minimum SDK of 16299 -- did you reduce the minimum SDK in the projects, or move to RS4 on the HoloLens? Thanks (and apologies in advance for the flurry of questions sure to come in the future from me as I struggle to get it all working on my end).

[jamescadd]

I have only tested on RS4 so no feedback or prior OS.

---

From: Daniel Andersen notifications@github.com Sent: Friday, April 27, 2018 3:45:44 PM To: Microsoft/WebRTC-universal-samples Cc: James Cadd; Mention Subject: Re: [Microsoft/WebRTC-universal-samples] Mixed Reality Capture for HoloLens (#6)

Hi, I am looking to use the Mosa/20180320-peercc-unity branch and seeing that @vgibilmannohttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvgibilmanno&data=02%7C01%7Cjacadd%40microsoft.com%7C495837e7766743e529f808d5ac909c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636604659457196788&sdata=EHnhUWeYFouIZMyVEZy6gRQEZuuz89%2B1BVPmewAZO5Y%3D&reserved=0 has successfully gotten MRC with WebRTC from HoloLens to desktop is very encouraging. I was curious: did you have to upgrade your HoloLens to the RS4 preview or were you able to hold off? I notice that the VS2017 projects in all the code are requiring a minimum SDK of 16299 -- did you reduce the minimum SDK in the projects, or move to RS4 on the HoloLens? Thanks (and apologies in advance for the flurry of questions sure to come in the future from me as I struggle to get it all working on my end).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2FWebRTC-universal-samples%2Fissues%2F6%23issuecomment-385112546&data=02%7C01%7Cjacadd%40microsoft.com%7C495837e7766743e529f808d5ac909c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636604659457206797&sdata=0727ord0xC0nt%2BzYSlsHizkord6AYNZ2uoaGjgMFKIc%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACmlAgabUG2dPJH04yQ03eGeINbaD6LJks5ts5-YgaJpZM4MrnGL&data=02%7C01%7Cjacadd%40microsoft.com%7C495837e7766743e529f808d5ac909c96%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636604659457206797&sdata=6f8oziqbHc237iQfZsPE0s52CM24AAYVa1Ln36K8FjA%3D&reserved=0.

[g-javi]

@DanAndersen I managed to get it working on SDK version 14393, however I'm having issues rendering both media streams into the unity video components. I have yet to test on RS4, I'll update you once I've done it.

[vgibilmanno]

@DanAndersen like @gjavi1 said it should work with the SDK version 14393. I have a relatively old build on my HoloLens. I think it is the build 10.0.14393.0 but I can't tell for sure until Monday because I don't have the HoloLens with me. I'l edit this comment on Monday in case I got it wrong.

[DanAndersen]

Thanks -- I was able to get the peercc-unity branch built and deployed onto the HoloLens with RS4. I was able to send video to/from the HoloLens running PeerCCUnity (a Unity app appearing as a floating 2D window in my HoloLens space) and the PeerConnectionClient.WebRtc desktop app. However, I am encountering the severe slowdowns that presumably were reported earlier when running PeerCCUnity. Encoding quality is severely diminished, frame rate is extremely low, and latency is half a minute or so (despite all devices being connected to a high-speed university network). I am running PeerCCUnity in Release mode without any debugging from Visual Studio.

@vgibilmanno : is the Unity app you showed in your GIF part of the same branch, or is it a separate codebase you are working on? Any chance it can be shared? I'm curious if the fact that your project is a full 3D Unity app, rather than one 2D app rendering among many at the OS level, is a reason why you're able to have good performance.

[g-javi]

@jamescadd current project solution for PeerCC is broken, there is a hard coded path in the Assembly-CSharp.csproj file, check this line.

[vgibilmanno]

@DanAndersen You have to build the native WebRTC and the WebRTC wrapper in release x86 mode in order get a better performance. Open the WebRtc.sln in this directory for that. Then copy the built WebRTC assemblies in the plugins directory in your Unity project and reference them in the UWP project where your conductor and signaller resides in. The Unity app I used isn't part of any of these repositories. But you'll find the codebase here. You still have to make some small changes in order to get it to work. In case you wonder, you don't need to explicitly assign a microphone or a playout device as the current WebRTC code handles that already for you.

[g-javi]

I'm having issues rendering the video stream in unity using the MediaEngineUWP plugin, when should I call the RemotePlay() method. I've noticed that the play method is not even being called, currently I have this:

Plugin.LoadRemoteMediaStreamSource((MediaStreamSource)source);
Plugin.RemotePlay();

Can someone please give me feedback on this, thanks in advance.

[morosev]

@gjavi1 The hard coded path has been fixed. RemotePlay() is removed from Unity script. When this method is called before rendering starts, two rendering loops per MediaEngine object are created. It works quite fine if RemotePlay() is not called at all.

[DanAndersen]

@vgibilmanno Thanks for the guidance. I've been able to successfully take the plugin files generated by the WebRtcUnityD3D solution and get a (moderately) low-latency MRC streaming video from the HoloLens to the PeerCC desktop app. Still working on cobbling together my own version for my use case but it's heartening to be able to make progress.

[g-javi]

@vgibilmanno @DanAndersen did you use the MediaEngineUWP.dll plugin to render your video inside unity, using it just gives me a black screen. I manage to render the video inside unity using the TexturesUWP from this project but I'm getting some performance issues.

Edit: I found the issue, the appxmanifest (from the unity build) needs to consume WebRtcScheme.dll

[DanAndersen]

@gjavi1 My test app so far doesn't have the rendering of the video inside Unity (just sending video + MRC to a desktop XAML app). Eventually I'll need to do rendering of the video in Unity too so I'll have to work more on this.

[ghost]

Following the unexpected blocker I hit earlier (see above) I got a HoloLens client running in our Jitsi VideoBridge / WebRTC conferencing system. The issue after the HL update was around adding audio streams - I'll investigate this later - but when I set RTCMediaStreamConstraints.audioEnabled to false, it works again. The frame rate seems to be much improved by setting the HoloLens camera resolution to 896x504 15FPS. Happy to share info.

[jamescadd]

@morosev Could you set video capture default in the HL branch to this: 896x504 15FPS

[morosev]

@jamescadd Application chooses the closest video capability match to 896x504 15FPS.

[binxie33]

Hi,

I tried to use the Mosa/20180320-peercc-unity branch, and the preparing and building went fine. But when I deploy the PeerConnectionClientUnity project on a Hololens device, it always gives me the the following error:

FileNotFoundException: The specified module could not be found. (Exception from HRESULT: 0x8007007E) at System.StubHelpers.StubHelpers.GetWinRTFactoryObject(IntPtr pCPCMD) at Org.WebRtc.Media.CreateMedia() at PeerConnectionClient.Signalling.Conductor..ctor() at PeerConnectionClient.Signalling.Conductor.get_Instance() at ControlScript.Start() at ControlScript.$Invoke20(Int64 instance, Int64 args) at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64 args, IntPtr method) (Filename: Line: 0)

Is there anything I missed here?

Thanks in advanced.

[DanAndersen]

@binxie33 I am attempting to solve the same issue ( see https://github.com/webrtc-uwp/webrtc-uwp-sdk/issues/50 ). I have not yet figured out the exact cause and solution, but I recommend watching that issue for updates.

[jamescadd]

@binxie33 What version is your HL OS? Maybe that has something to do with it. I'm unable to reproduce this, but VS logs may help. Just to double check, everyone is using the D3D solution?

[DanAndersen]

@jamescadd I am running the D3D solution -- I start up the solution, build PeerConnectionClientUnity (which builds the other projects) and I'm able to deploy the app onto the HoloLens -- but the error whenever trying to run Org.WebRtc.Media.CreateMedia() is still present.

Could the issue be related to this commit ( https://github.com/webrtc-uwp/chromium-build/commit/7f63b210472ec618860d6a47fc7478720559d63a ) ? Because I'm trying to build for the HoloLens (x86), I think the intended family would be WINAPI_FAMILY_APP (that is what is listed in the preprocessor definitions of Org.WebRtc.Uwp.vcxproj. But if some of the code is being defined as WINAPI_FAMILY_DESKTOP_APP, that might explain some of the necessary libraries not being included, causing an issue when deploying onto the HoloLens.

[binxie33]

@jamescadd I tried on both RS4 (10.0.17134.1011) and the previous version (10.0.14393.2189) of Hololens devices, and got the same error. As @DanAndersen mentiond, deploying the app to hololens succeeded, but only when you run it, you will see the error "The specified module could not be found" in the VS debug output.

[binxie33]

@DanAndersen Thanks for the info. Good to know I am not the only one struggling with the issue. :) Is it possible for you to send me the previous build of binaries? I just need them for POC purpose. Thanks again!

[DanAndersen]

@binxie33 Here is a Dropbox link with the old DLLs that I had made from an earlier build (before this issue arose for me). https://www.dropbox.com/sh/u3ag71x2e6qnzqc/AAB5j9JR8N3uUaG00YrNg_xSa?dl=0

[jamescadd]

@morosev do you think the commit above could be causing the dll load issue? @danandersen are there any steps beyond launching the app required to repro? I haven't run with debugger attached yet but will check the logs next time.

---

From: Daniel Andersen notifications@github.com Sent: Saturday, May 19, 2018 10:09:47 PM To: Microsoft/WebRTC-universal-samples Cc: James Cadd; Mention Subject: Re: [Microsoft/WebRTC-universal-samples] Mixed Reality Capture for HoloLens (#6)

@binxie33https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbinxie33&data=02%7C01%7Cjacadd%40microsoft.com%7C390930f168164729c14508d5be0fe871%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636623897893675459&sdata=zSYbWDq%2B5jCkC%2FmnZtfg5qX5ZBlOrcrQOI53QeVAj0U%3D&reserved=0 Here is a Dropbox link with the old DLLs that I had made from an earlier build (before this issue arose for me). https://www.dropbox.com/sh/u3ag71x2e6qnzqc/AAB5j9JR8N3uUaG00YrNg_xSa?dl=0https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.dropbox.com%2Fsh%2Fu3ag71x2e6qnzqc%2FAAB5j9JR8N3uUaG00YrNg_xSa%3Fdl%3D0&data=02%7C01%7Cjacadd%40microsoft.com%7C390930f168164729c14508d5be0fe871%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636623897893675459&sdata=x5XRSMMW%2FA%2Bo5%2FdOJFtYtakG63QWiOsvoOGNzq02jKQ%3D&reserved=0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2FWebRTC-universal-samples%2Fissues%2F6%23issuecomment-390458319&data=02%7C01%7Cjacadd%40microsoft.com%7C390930f168164729c14508d5be0fe871%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636623897893685464&sdata=3Yk9%2BVcb7ZezfWXSllM9JP7okBq7INaZpafz0jJxiP8%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACmlAs9KG_g2iRjTSpo6Y6OS0g4TcYA5ks5t0PqbgaJpZM4MrnGL&data=02%7C01%7Cjacadd%40microsoft.com%7C390930f168164729c14508d5be0fe871%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636623897893685464&sdata=6xKAC2msRz6X%2Bd0TaGcMiICEmPTzev75t6TB0TOkT0U%3D&reserved=0.

[DanAndersen]

@jamescadd There shouldn't be any additional steps to reproduce, as the call to Org.WebRtc.Media.CreateMedia() happens in the constructor of the Conductor class (which is initialized in a Start() script). The debug logs should end up showing it.

My steps for reproducing were just:

git clone --recursive -b Mosa/20180320-peercc-unity https://github.com/ortclib/ortclib-sdk.git mosa_fresh
cd mosa_fresh
bin\prepare.bat -loglevel 4

Then opening WebRtcUnityD3D.sln, building all projects including PeerConnectionClientUnity, then deploying it onto the HoloLens and and attaching the debugger. The debug logs will print the Failed to load 'Org.WebRtc.dll', expected x86 architecture, but was Unknown architecture. You must recompile your plugin for x86 architecture. error while the app is starting up, when the Unity logo is appearing before the 3D scene loads.

EDIT: The issue is able to be replicated when using the HoloLens Emulator (10.0.14393.1358) as well. Deploying PeerConnectionClientUnity to the emulator will yield the same errors about the failure of Org.WebRtc.dll to load. Even though the emulator doesn't have a camera, the DLL should at least be loading and the CreateMedia() function should be able to be called.

[DanAndersen]

@binxie33 I was able to revert to an earlier commit which still works on the 16299 SDK. This should at least be workable for the time being. See https://github.com/webrtc-uwp/webrtc-uwp-sdk/issues/50#issuecomment-390538189 for details.

[binxie33]

@DanAndersen Good finding! I will give it a try.

[morosev]

@DanAndersen @binxie33 @jamescadd I succeeded to reproduce the problem on HoloLens emulator 17134. Actually, it is running well on one emulator where the problem related to webrtc library loading is not expressed, On the other hand, when the application is started on emulator installed on another machine it fails with a message "Failed to load 'Org.WebRtc.dll', expected x86 architecture, but was Unknown architecture. You must recompile your plugin for x86 architecture." Since Org.WebRtc.dll could be loaded in the first case, I will try to find a difference between two configurations. Keeping you updated about the progress...

[fcagnetta]

Hello there, I'm trying to set up PeerCC in order to open a communication between a hololens (Unity app) and a PC. The Unity app is shown correctly, but it's impossible to connect to the server and I can't figure out why. I get these messages in output-debug in VS:

_Failed to get spatial stage statics - can't retrieve or interact with boundaries! Error code: '0x80040154'.

(Filename: C:\buildslave\unity\build\Runtime/VR/HoloLens/StageRoot.cpp Line: 20)

'PeerCCUnity.exe' (CoreCLR: CoreCLR_UWP_Domain): caricamento di 'C:\Data\Users\DefaultAccount\AppData\Local\DevelopmentFiles\PeerCCUnityVS.Releasex86.Utente11\UnityEngine.dll' completato. Impossibile trovare o aprire il file PDB. Exception thrown: 'System.IO.FileNotFoundException' in PeerConnectionClientCore.dll Il thread 0x16d0 è terminato con il codice 0 (0x0). Il thread 0x1754 è terminato con il codice 0 (0x0). Il programma '[5376] PeerCCUnity.exe' è terminato con il codice -1 (0xffffffff).

I'm using Unity 2017.4.1f1, VS2017. The Unity PeerCC app is the one from Mosa/20180320-peercc-unity repo. Do you have some idea on it?