I followed instructions and finally came up with WebRTC solution.
However, the whole solution didn't provide compatible PeerConnectionUnityD3DCore solution for Unity use, so I took it from branch: https://github.com/Microsoft/WebRTC-universal-samples.git.
There were something I figured out:
The Webrtc library which were built from https://github.com/ortclib/ortclib-sdk.git didn't compatible with PeerConnectionUnityD3DCore solution because of missing or modified methods of which it provided. Eg:
1) WebRTC.StartTracing method:
In Conductor.cs (PeerConnectionUnityD3DCore): WebRTC.StartTracing("webrtc-trace.txt");
Method provided in built WebRtc lib: WebRTC.StartTracing();
2) Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat method:
In Conductor.cs (PeerConnectionUnityD3DCore): Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat(
(int)VideoCaptureProfile.Width, (int)VideoCaptureProfile.Height, (int)VideoCaptureProfile.FrameRate, VideoCaptureProfile.MrcEnabled);
Method provided in built WebRtc lib:
Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat(
(int)VideoCaptureProfile.Width, (int)VideoCaptureProfile.Height, (int)VideoCaptureProfile.FrameRate);
3) Media.AddVideoTrackMediaElementPair & Media.RemoveVideoTrackMediaElementPair:
Built WebRtc lib didn't provide these methods but PeerConnectionUnityD3DCore (Conductor.cs) called them.
4) Media.CreateMedia().CreateMediaSource or Media.CreateMedia().CreateMediaStreamSource: methods
In Conductor.cs (PeerConnectionUnityD3DCore):
Media.CreateMedia().CreateMediaStreamSource(_selfVideoTrack, type, "SELF");
Method provided in built WebRtc lib:
Media.CreateMedia().CreateMediaSource(_selfVideoTrack, "SELF");
5) Org.WebRtc.CodecInfo method:
In Conductor.cs (PeerConnectionUnityD3DCore):
Org.WebRtc.CodecInfo( int ClockRate, string Name)
Method provided in built WebRtc lib:
Org.WebRtc.CodecInfo( int Id, int ClockRate, string Name)
Because of these differences, I tried to modify PeerConnectionUnityD3DCore (mainly Conductor.cs) to be compatible with built WebRtc lib then deployed to Hololens.
After connected to a desktop peer the app threw exception as below:
Exception thrown: 'System.InvalidCastException' in Assembly-CSharp.dll
Exception thrown: 'System.InvalidCastException' in Assembly-CSharp.dll
Unable to cast object of type 'System.__ComObject' to type 'Windows.Media.Core.MediaStreamSource'.
after a few seconds, the app crashed with log as below:
The program '[2868] PeerCCUnity1.exe' has exited with code -1073741189 (0xc000027b).
Summary:
I figured out that there could be many different WebRtc versions. But after having a look on PeerConnectionUnityD3DCore, I think nuget WebRtc ver 1.62.1.1-Beta is the most suitable, However I cannot get .dll and .winmd files from nuget package. Thus, I will try to modify the WebRtc solution (from https://github.com/ortclib/ortclib-sdk.git branch) to get it compatible with PeerConnectionUnityD3DCore and fix the exception that I met.
Moreover, I would be very appreciated if you can provide me source code or .dll and .winmd files of WebRtc Nuget package ver 1.62.1.1-Beta, or any source of information that I can use to modify WebRtc solution myself.
@jamescadd I would like to response my result when playing with branch: https://github.com/ortclib/ortclib-sdk.git
I followed instructions and finally came up with WebRTC solution. However, the whole solution didn't provide compatible PeerConnectionUnityD3DCore solution for Unity use, so I took it from branch: https://github.com/Microsoft/WebRTC-universal-samples.git.
There were something I figured out:
2) Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat method: In Conductor.cs (PeerConnectionUnityD3DCore): Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat( (int)VideoCaptureProfile.Width, (int)VideoCaptureProfile.Height, (int)VideoCaptureProfile.FrameRate, VideoCaptureProfile.MrcEnabled); Method provided in built WebRtc lib: Org.WebRtc.WebRTC.SetPreferredVideoCaptureFormat( (int)VideoCaptureProfile.Width, (int)VideoCaptureProfile.Height, (int)VideoCaptureProfile.FrameRate);
3) Media.AddVideoTrackMediaElementPair & Media.RemoveVideoTrackMediaElementPair: Built WebRtc lib didn't provide these methods but PeerConnectionUnityD3DCore (Conductor.cs) called them.
4) Media.CreateMedia().CreateMediaSource or Media.CreateMedia().CreateMediaStreamSource: methods In Conductor.cs (PeerConnectionUnityD3DCore): Media.CreateMedia().CreateMediaStreamSource(_selfVideoTrack, type, "SELF"); Method provided in built WebRtc lib: Media.CreateMedia().CreateMediaSource(_selfVideoTrack, "SELF");
5) Org.WebRtc.CodecInfo method: In Conductor.cs (PeerConnectionUnityD3DCore): Org.WebRtc.CodecInfo( int ClockRate, string Name) Method provided in built WebRtc lib: Org.WebRtc.CodecInfo( int Id, int ClockRate, string Name)
Because of these differences, I tried to modify PeerConnectionUnityD3DCore (mainly Conductor.cs) to be compatible with built WebRtc lib then deployed to Hololens. After connected to a desktop peer the app threw exception as below:
Exception thrown: 'System.InvalidCastException' in Assembly-CSharp.dll Exception thrown: 'System.InvalidCastException' in Assembly-CSharp.dll Unable to cast object of type 'System.__ComObject' to type 'Windows.Media.Core.MediaStreamSource'.
after a few seconds, the app crashed with log as below: The program '[2868] PeerCCUnity1.exe' has exited with code -1073741189 (0xc000027b).
Summary: I figured out that there could be many different WebRtc versions. But after having a look on PeerConnectionUnityD3DCore, I think nuget WebRtc ver 1.62.1.1-Beta is the most suitable, However I cannot get .dll and .winmd files from nuget package. Thus, I will try to modify the WebRtc solution (from https://github.com/ortclib/ortclib-sdk.git branch) to get it compatible with PeerConnectionUnityD3DCore and fix the exception that I met. Moreover, I would be very appreciated if you can provide me source code or .dll and .winmd files of WebRtc Nuget package ver 1.62.1.1-Beta, or any source of information that I can use to modify WebRtc solution myself.