microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
908 stars 282 forks source link

VideoChatDemo data transfer failed #43

Open derekfreed opened 5 years ago

derekfreed commented 5 years ago

I've been trying to get the VideoChatDemo scene deployed to 2 clients and was able to get them both connected but once I tried to create the offer I didn't see any data transfered. The server printed out a lot of info that I'm not sure what to make of:

dss GET /data/123 404 - - 0.203 ms +409ms dss GET /data/321 404 - - 0.112 ms +122ms dss:body {"MessageType":1,"Data":"v=0\r\no=- 8056751785094099368 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE audio video\r\na=msid-semantic: WMS local_av_stream\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:3meE\r\na=ice-pwd:P2HnrFWRx5F/alVQdmKNIP36\r\na=ice-options:trickle\r\na=fingerprint:sha-256 7F:EB:C6:03:42:88:18:C4:97:34:00:F3:62:27:54:5D:C6:92:15:0D:F9:77:6C:FD:9D:3F:B8:C2:CF:C6:4E:1A\r\na=setup:actpass\r\na=mid:audio\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=sendrecv\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:102 ILBC/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:2387358855 cname:Uqgu1Ku8ixh5j3qk\r\na=ssrc:2387358855 msid:local_av_stream local_audio\r\na=ssrc:2387358855 mslabel:local_av_stream\r\na=ssrc:2387358855 label:local_audio\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 124 125\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:3meE\r\na=ice-pwd:P2HnrFWRx5F/alVQdmKNIP36\r\na=ice-options:trickle\r\na=fingerprint:sha-256 7F:EB:C6:03:42:88:18:C4:97:34:00:F3:62:27:54:5D:C6:92:15:0D:F9:77:6C:FD:9D:3F:B8:C2:CF:C6:4E:1A\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:10 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07\r\na=sendrecv\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 x-google-profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 multiplex/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 acn=VP9;x-google-profile-id=0\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:127 red/90000\r\na=rtpmap:124 rtx/90000\r\na=fmtp:124 apt=127\r\na=rtpmap:125 ulpfec/90000\r\na=ssrc-group:FID 3584077471 988498931\r\na=ssrc:3584077471 cname:Uqgu1Ku8ixh5j3qk\r\na=ssrc:3584077471 msid:local_av_stream local_video\r\na=ssrc:3584077471 mslabel:local_av_stream\r\na=ssrc:3584077471 label:local_video\r\na=ssrc:988498931 cname:Uqgu1Ku8ixh5j3qk\r\na=ssrc:988498931 msid:local_av_stream local_video\r\na=ssrc:988498931 mslabel:local_av_stream\r\na=ssrc:988498931 label:local_video\r\n","IceDataSeparator":"|"} +0ms dss POST /data/123 200 - - 37.884 ms +272ms dss:body {"MessageType":3,"Data":"candidate:995267180 1 udp 2122063615 10.88.68.134 57769 typ host generation 0 ufrag 3meE network-id 4 network-cost 10|1|video","IceDataSeparator":"|"} +41ms dss POST /data/123 200 - - 5.998 ms +13ms dss:body {"MessageType":3,"Data":"candidate:1154973090 1 udp 2122197247 2001:4898:e0:2077:c409:46b6:64ab:b4d4 57763 typ host generation 0 ufrag 3meE network-id 6 network-cost 10|0|audio","IceDataSeparator":"|"} +11ms dss POST /data/123 200 - - 5.801 ms +12ms dss:body {"MessageType":3,"Data":"candidate:1203989023 1 udp 2122262783 2001:4898:e0:2077:810b:d91f:8e56:3b05 57762 typ host generation 0 ufrag 3meE network-id 5 network-cost 10|0|audio","IceDataSeparator":"|"} +10ms dss POST /data/123 200 - - 3.194 ms +8ms dss:body {"MessageType":3,"Data":"candidate:77142221 1 udp 2122129151 192.168.137.1 57764 typ host generation 0 ufrag 3meE network-id 1 network-cost 10|0|audio","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 5.917 ms +10ms dss:body {"MessageType":3,"Data":"candidate:1154973090 1 udp 2122197247 2001:4898:e0:2077:c409:46b6:64ab:b4d4 57767 typ host generation 0 ufrag 3meE network-id 6 network-cost 10|1|video","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 3.164 ms +6ms dss:body {"MessageType":3,"Data":"candidate:995267180 1 udp 2122063615 10.88.68.134 57765 typ host generation 0 ufrag 3meE network-id 4 network-cost 10|0|audio","IceDataSeparator":"|"} +7ms dss POST /data/123 200 - - 6.374 ms +9ms dss:body {"MessageType":3,"Data":"candidate:77142221 1 udp 2122129151 192.168.137.1 57768 typ host generation 0 ufrag 3meE network-id 1 network-cost 10|1|video","IceDataSeparator":"|"} +9ms dss POST /data/123 200 - - 2.984 ms +7ms dss:body {"MessageType":3,"Data":"candidate:1203989023 1 udp 2122262783 2001:4898:e0:2077:810b:d91f:8e56:3b05 57766 typ host generation 0 ufrag 3meE network-id 5 network-cost 10|1|video","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 7.342 ms +11ms dss:body {"MessageType":3,"Data":"candidate:155227887 1 tcp 1518283007 2001:4898:e0:2077:810b:d91f:8e56:3b05 53518 typ host tcptype passive generation 0 ufrag 3meE network-id 5 network-cost 10|0|audio","IceDataSeparator":"|"} +10ms dss POST /data/123 200 - - 2.401 ms +6ms dss:body {"MessageType":3,"Data":"candidate:173310290 1 tcp 1518217471 2001:4898:e0:2077:c409:46b6:64ab:b4d4 53519 typ host tcptype passive generation 0 ufrag 3meE network-id 6 network-cost 10|0|audio","IceDataSeparator":"|"} +11ms dss POST /data/123 200 - - 5.101 ms +13ms dss:body {"MessageType":3,"Data":"candidate:1243276349 1 tcp 1518149375 192.168.137.1 53520 typ host tcptype passive generation 0 ufrag 3meE network-id 1 network-cost 10|0|audio","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 3.884 ms +7ms dss:body {"MessageType":3,"Data":"candidate:155227887 1 tcp 1518283007 2001:4898:e0:2077:810b:d91f:8e56:3b05 53522 typ host tcptype passive generation 0 ufrag 3meE network-id 5 network-cost 10|1|video","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 5.332 ms +9ms dss:body {"MessageType":3,"Data":"candidate:1976659612 1 tcp 1518083839 10.88.68.134 53521 typ host tcptype passive generation 0 ufrag 3meE network-id 4 network-cost 10|0|audio","IceDataSeparator":"|"} +7ms dss POST /data/123 200 - - 2.332 ms +5ms dss:body {"MessageType":3,"Data":"candidate:1243276349 1 tcp 1518149375 192.168.137.1 53524 typ host tcptype passive generation 0 ufrag 3meE network-id 1 network-cost 10|1|video","IceDataSeparator":"|"} +6ms dss POST /data/123 200 - - 6.486 ms +10ms dss:body {"MessageType":3,"Data":"candidate:173310290 1 tcp 1518217471 2001:4898:e0:2077:c409:46b6:64ab:b4d4 53523 typ host tcptype passive generation 0 ufrag 3meE network-id 6 network-cost 10|1|video","IceDataSeparator":"|"} +11ms dss POST /data/123 200 - - 3.461 ms +7ms dss:body {"MessageType":3,"Data":"candidate:1976659612 1 tcp 1518083839 10.88.68.134 53529 typ host tcptype passive generation 0 ufrag 3meE network-id 4 network-cost 10|1|video","IceDataSeparator":"|"} +8ms dss POST /data/123 200 - - 7.485 ms +12ms dss GET /data/123 200 - - 0.433 ms +5ms dss GET /data/321 404 - - 0.169 ms +112ms

djee-ms commented 5 years ago

Hi @derekfreed,

From the log we can tell that:

After that for some reason the other peer is not responding with an answer (MessageType=2).

Note that the log doesn't provide information about which peer is doing that; it's whatever peer asked "here is a message to deliver to 123" and "is there any message for 123?", respectively, which in the former case may be 123 itself if misconfigured. I'd suggest to make sure that both peers are configured correctly with each other's ID as their RemotePeerId. Either the peer getting the offer is the one who submitted it (same local and remote ID), or the peer getting the offer is trying to answer with a wrong ID, so it doesn't get delivered to the other peer. Unfortunately there is no way to tell from the logs currently, this should probably be improved for debugging.

For Unity apps, you need to configure the RemotePeerId field as described in the tutorial. In particular note that because the RemotePeerId is serialized by Unity, you need to set it once, generate a solution and deploy to the first device, then change it to the other peer's ID, and re-generate the solution and deploy to the other device. You cannot deploy the same build on both devices, otherwise they will have the same serialized value for RemotePeerId.

derekfreed commented 5 years ago

Thanks for the response, I did what you described above as far as making different builds with the respective localID and RemotePeerId. Perhaps I messed it up though, so I'll try at it again.

djee-ms commented 4 years ago

@derekfreed is there any update on that issue?

zhuangce commented 4 years ago

i find the same error with derekfreed ,here is my log: dss GET /data/SKY-20180410DFD 404 - - 0.045 ms +402ms dss GET /data/HOLOLENS-25NKM 404 - - 0.046 ms +129ms dss:body {"MessageType":1,"Data":"v=0\r\no=- 3557513225521305901 3 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS local_av_stream\r\nm=audio 57843 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 192.168.0.132\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:2460416669 1 udp 2122260223 192.168.0.132 57843 typ host generation 0 network-id 1\r\na=candidate:3693413997 1 tcp 1518280447 192.168.0.132 55045 typ host tcptype passive generation 0 network-id 1\r\na=ice-ufrag:JnB5\r\na=ice-pwd:mRjmaP72P0MX6nA/MSne0pPM\r\na=ice-options:trickle\r\na=fingerprint:sha-256 EA:57:94:B9:7B:93:7E:26:72:8D:F2:05:AF:C1:9E:29:39:0F:E8:58:1F:8A:F0:FD:41:DD:8C:9D:18:5D:D1:82\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=sendrecv\r\na=msid:local_av_stream local_audio\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:102 ILBC/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:1889206609 cname:lIiqxtHDzGcEWplc\r\na=ssrc:1889206609 msid:local_av_stream local_audio\r\na=ssrc:1889206609 mslabel:local_av_stream\r\na=ssrc:1889206609 label:local_audio\r\nm=video 57844 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 124 125\r\nc=IN IP4 192.168.0.132\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:2460416669 1 udp 2122260223 192.168.0.132 57844 typ host generation 0 network-id 1\r\na=candidate:3693413997 1 tcp 1518280447 192.168.0.132 55046 typ host tcptype passive generation 0 network-id 1\r\na=ice-ufrag:JnB5\r\na=ice-pwd:mRjmaP72P0MX6nA/MSne0pPM\r\na=ice-options:trickle\r\na=fingerprint:sha-256 EA:57:94:B9:7B:93:7E:26:72:8D:F2:05:AF:C1:9E:29:39:0F:E8:58:1F:8A:F0:FD:41:DD:8C:9D:18:5D:D1:82\r\na=setup:actpass\r\na=mid:1\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:10 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 x-google-profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 multiplex/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 acn=VP9;x-google-profile-id=0\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:127 red/90000\r\na=rtpmap:124 rtx/90000\r\na=fmtp:124 apt=127\r\na=rtpmap:125 ulpfec/90000\r\n","IceDataSeparator":""} +18s dss POST /data/HOLOLENS-25NKM 200 - - 4.644 ms +159ms dss GET /data/SKY-20180410DFD 404 - - 0.046 ms +242ms dss GET /data/HOLOLENS-25NKM 200 - - 0.068 ms +138ms

zhuangce commented 4 years ago

@djee-ms i've been hande this problem for a long time ,can you help find out what's the reason?

djee-ms commented 4 years ago

@zhuangce : Same answer, your logs show an offer SDP message with VP8 video and OPUS audio, sent from one peer, and received by the other. Note that the audio offer is bidirectional (a=sendrecv), but the video offer is just to receive some remote video and not to send anything (a=recvonly), is that expected for your use case? I can't tell much more from this unfortunately.

zhuangce commented 4 years ago

@djee-ms i solved this problem by change the remote / local id to 123/456 ,i wasn't sure if it is cause by some illegal character in id