search

microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
909 stars 283 forks source link

There is no display of local video on hololens1 and hololens2 #285

Closed Lenovezhou closed 4 years ago

Lenovezhou commented 4 years ago

Describe the bug There is no display of local video on hololens1 and hololens2

To Reproduce Steps to reproduce the behavior:

  1. Compiling DLL with vs2019
  2. Use unity2018.4.15 to open the project： MixedReality-WebRTC\libs\Microsoft.MixedReality.WebRTC.Unity
  3. Operation signaling server
  4. Deploy and run on hololens1 and hololens2

Expected behavior I want to see localvideo on hololens and display it in remotevide of unityeditor

SDP messages Received SDP message: type=Answer data=v=0 o=- 5716825584108628144 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 1 a=msid-semantic: WMS local_av_stream m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:i8Ur a=ice-pwd:jMAX9oI+ZWFb05HAZF4KGEL5 a=ice-options:trickle a=fingerprint:sha-256 7B:DF:94:72:68:0F:15:1F:9E:20:EC:1A:A5:F4:BA:37:11:97:FC:6B:2B:9C:41:C6:87:1B:13:4E:19:2C:40:2B a=setup:active a=mid:0 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid a=sendrecv a=msid:local_av_stream local_audio a=rtcp-mux a=rtpmap:111 opus/48000/2 a=rtcp-fb:111 transport-cc a=fmtp:111 minptime=10;useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:102 ILBC/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 a=rtpmap:13 CN/8000 a=rtpmap:110 telephone-event/48000 a=rtpmap:112 telephone-event/32000 a=rtpmap:113 telephone-event/16000 a=rtpmap:126 telephone-event/8000 a=ssrc:1397576014 cname:gmj4/ebvcY3gHA4/ m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 127 124 125 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:i8Ur a=ice-pwd:jMAX9oI+ZWFb05HAZF4KGEL5 a=ice-options:trickle a=fingerprint:sha-256 7B:DF:94:72:68:0F:15:1F:9E:20:EC:1A:A5:F4:BA:37:11:97:FC:6B:2B:9C:41:C6:87:1B:13:4E:19:2C:40:2B a=setup:active a=mid:1 a=extmap:2 urn:ietf:params:rtp-hdrext:toffset a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:4 urn:3gpp:video-orientation a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:10 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07 a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid a=recvonly a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=rtpmap:98 VP9/90000 a=rtcp-fb:98 goog-remb a=rtcp-fb:98 transport-cc a=rtcp-fb:98 ccm fir a=rtcp-fb:98 nack a=rtcp-fb:98 nack pli a=fmtp:98 x-google-profile-id=0 a=rtpmap:99 rtx/90000 a=fmtp:99 apt=98 a=rtpmap:127 red/90000 a=rtpmap:124 rtx/90000 a=fmtp:124 apt=127 a=rtpmap:125 ulpfec/90000

UnityEngine.Debug:Log(Object) Microsoft.MixedReality.WebRTC.Unity.d__15:MoveNext() (at Assets/Microsoft.MixedReality.WebRTC.Unity/Scripts/Signaling/NodeDssSignaler.cs:227) UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

Screenshots HoloLens 20200420_225249_HoloLens

UnityEditor Snipaste_2020-04-21_13-52-59

Environment

HenryCui1979 commented 4 years ago

I met with same issue like you.The sdp set "recvonly" for video.Do you solve this problem now?

djee-ms commented 4 years ago

The SDP has no effect on displaying the local video; the local video display doesn't need an established WebRTC connection to be displayed. Of course the reason the local HoloLens video doesn't appear on the remote peer is because of RecvOnly, but that is because there's no local video in the first place to send.

If the local video doesn't show up when you start the VideoChatDemo sample in HoloLens 1/2, then something went wrong with opening the camera, most likely a permission issue. You should check in the Settings of Windows 10 that the demo app has been authorized to access the camera and microphone of the HoloLens device. Normally on the first run of the app the OS will display a dialog asking for user permission, but I have seen a bug that when debugging sometimes this dialog is skipped and the permission denied. So you need to manually re-authorize the app in Settings > Privacy > Camera/Microphone.

Lenovezhou commented 4 years ago

The SDP has no effect on displaying the local video; the local video display doesn't need an established WebRTC connection to be displayed. Of course the reason the local HoloLens video doesn't appear on the remote peer is because of RecvOnly, but that is because there's no local video in the first place to send. If the local video doesn't show up when you start the VideoChatDemo sample in HoloLens 1/2, then something went wrong with opening the camera, most likely a permission issue. You should check in the Settings of Windows 10 that the demo app has been authorized to access the camera and microphone of the HoloLens device. Normally on the first run of the app the OS will display a dialog asking for user permission, but I have seen a bug that when debugging sometimes this dialog is skipped and the permission denied. So you need to manually re-authorize the app in Settings > Privacy > Camera/Microphone. I'm pretty sure I authorized the camera and microphone permissions. image

Also, when the app first launched the app, I agreed to use the camera and microphone. I also confirmed in the settings. And, I can hear the voice from UnityEditor. UnityEditor can also hear my voice from HoloLens.

Lenovezhou commented 4 years ago

I met with same issue like you.The sdp set "recvonly" for video.Do you solve this problem now? hello @HenryCui1979 no,I can't I ask someone solved this problem at this link,You can follow him: https://github.com/microsoft/MixedReality-WebRTC/issues/235#issuecomment-615833782

Lenovezhou commented 4 years ago

I met with same issue like you.The sdp set "recvonly" for video.Do you solve this problem now?

hi @HenryCui1979 I used DLL in C × nuget packages to solve this problem.And Run it on HoloLens2 。 You can refer to this link。 https://microsoft.github.io/MixedReality-WebRTC/manual/installation.html