search

AirenSoft / OvenMediaEngine

OvenMediaEngine (OME) is a Sub-Second Latency Live Streaming Server with Large-Scale and High-Definition. #WebRTC #LLHLS
https://airensoft.com/ome.html
GNU Affero General Public License v3.0
2.58k stars 1.06k forks source link

UDP not working in WebRTC publisher #699

Closed DScrp closed 2 years ago

DScrp commented 2 years ago

Hello. I used OME 12 before and could switch between UDP and TCP protocols. The UDP protocol was enabled by default and if i want enable TCP, i attaching the query transport=tcp to the existing WebRTC play URL. Everything worked fine. But now I installed OME 13.1 and can't switch to UDP.

When I use the following config everything works fine via TCP:

<TcpRelay>*:3478</TcpRelay>
<TcpForce>true</TcpForce>
<IceCandidate>*:10000-10005/udp</IceCandidate>
<TcpRelayWorkerCount>4</TcpRelayWorkerCount>

But the connection takes a long time and I would like to speed it up, as I did earlier on OME 12 and UDP protocol. But when I set TcpForce to false or delete this setting from Server.xml, the stream stops loading in player. I try it in OvenMediaPlayer, in my player and in https://demo.ovenplayer.com/ With TcpForce=true works fine. If change it to false - not working.

Logs:

[2022-03-15 21:57:23.229] I [SPRtcSig-T3334:3040106] Signalling | rtc_signalling_server.cpp:232  | New client is connected: <ClientSocket: 0x7f793004abd0, #39, Connected, TCP, Nonblocking, 123.123.123.123:8007>
[2022-03-15 21:57:23.387] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(10)/Stream total(10)/App total(10)
[2022-03-15 21:57:23.490] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:9020 / Packet type : 0 GateType : 0
[2022-03-15 21:57:33.411] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: pWvW (ufrag in peer SDP: TcOQ)
[2022-03-15 21:57:33.411] I [SPICE-U10002:3040112] ICE | ice_port.cpp:783  | Update the client to the port list: to 123.123.123.123:7954 from 123.123.123.123:9020
[2022-03-15 21:57:33.421] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(11)/Stream total(11)/App total(11)
[2022-03-15 21:57:43.442] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: Xhg5 (ufrag in peer SDP: pWvW)
[2022-03-15 21:57:43.442] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:9078 / Packet type : 0 GateType : 0
[2022-03-15 21:57:43.451] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(12)/Stream total(12)/App total(12)
[2022-03-15 21:57:53.742] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: BDoh (ufrag in peer SDP: Xhg5)
[2022-03-15 21:57:53.742] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:7960 / Packet type : 0 GateType : 0
[2022-03-15 21:57:53.752] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(13)/Stream total(13)/App total(13)
[2022-03-15 21:58:04.744] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: tnqY (ufrag in peer SDP: BDoh)
[2022-03-15 21:58:04.744] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:7945 / Packet type : 0 GateType : 0
[2022-03-15 21:58:04.753] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(14)/Stream total(14)/App total(14)
[2022-03-15 21:58:04.796] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:7945 / Packet type : 0 GateType : 0
[2022-03-15 21:58:15.741] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: o8a8 (ufrag in peer SDP: tnqY)
[2022-03-15 21:58:15.741] I [SPICE-U10002:3040112] ICE | ice_port.cpp:783  | Update the client to the port list: to 123.123.123.123:9720 from 123.123.123.123:7945
[2022-03-15 21:58:15.748] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(15)/Stream total(15)/App total(15)
[2022-03-15 21:58:26.744] W [SPICE-U10002:3040112] ICE | ice_port.cpp:745  | Mismatched ufrag: gq5O (ufrag in peer SDP: o8a8)
[2022-03-15 21:58:26.744] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:9724 / Packet type : 0 GateType : 0
[2022-03-15 21:58:26.753] I [SPRtcSig-T3334:3040106] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-1/channel-3 on the WebRTC publisher. WebRTC(16)/Stream total(16)/App total(16)
[2022-03-15 21:58:26.803] I [SPICE-U10002:3040112] ICE | ice_port.cpp:779  | Add the client to the port list: 123.123.123.123:9724 / Packet type : 0 GateType : 0

How can I use UDP in my streams like I did in OME 12 and increase connection speed?

getroot commented 2 years ago

I tested the WebRTC/udp playback and confirmed that it works fine. Check if the UDP port is open properly.

And in my opinion, there is no big difference in connection speed between WebRTC/udp and WebRTC/tcp. The only difference is due to TCP slow start, which does not cause human perceptible initial connection delay (tens of milliseconds). If it's really because of tcp slow start, try changing the Congestion Control method to BBR.

https://airensoft.gitbook.io/ovenmediaengine/troubleshooting#5-3.-congestion-control-change-to-bbr

As you know, UDP does not have good quality in high definition because there are many environments with high packet loss.

DScrp commented 2 years ago

Now I understand the problem. If I do not use video re-encoding, then everything works with UDP and TCP:

/usr/local/bin/ffmpeg -hide_banner -use_wallclock_as_timestamps 1 -stream_loop 0 -rtsp_transport udp -i rtsp://admin:password@192.168.1.101/Streaming/Channels/3 -vcodec copy -af "aresample=async=1:first_pts=0" -acodec aac -f flv -flvflags no_duration_filesize "rtmp://my.site:1936/camera-1/channel-3";

If i use video re-encoding, works only with TCP:

/usr/local/bin/ffmpeg -hide_banner -hwaccel cuda -use_wallclock_as_timestamps 1 -stream_loop 0 -rtsp_transport udp -i rtsp://admin:password@192.168.1.101/Streaming/Channels/3 -c:v h264_nvenc -b:v 2048k -bf 0 -af "aresample=async=1:first_pts=0" -acodec aac -f flv -flvflags no_duration_filesize "rtmp://my.site:1936/camera-1/channel-3";

What do I need to do to make everything work with UDP and TCP with re-encoding? Unfortunately, I need to use re-encoding. And according to my tests, the connection via UDP is established several times faster than via TCP. Therefore, I need to use re-encoding and the UDP protocol.

getroot commented 2 years ago

I can only guess the cause of your problem.

DScrp commented 2 years ago

Thank you. But now there is another problem. In desktop browsers and Android work fine with TCP and UDP. But in iOS video is not loading. Two same errors in console: Websocket network error: The operation couldn'r be completed (OSStatus error - 9807). Websocket network error: The operation couldn'r be completed (OSStatus error - 9807).

https://www.osstatus.com/search/results?platform=all&framework=all&search=9807 - about this errors on Apple site.

In Server.xml there are all SSL certificates in accordance with the manual https://airensoft.gitbook.io/ovenmediaengine/streaming/tls-encryption Tested on two different servers. I use SSL from Let's Encrypt.

getroot commented 2 years ago

This is an issue that cannot be reproduced in my testbed. If you ever set up a certificate, check if the Chain Cert and Domain Cert are correctly set separately. Does the Chain Cert contain a Domain Cert, or does the Domain Cert contain a Chain Cert?

DScrp commented 2 years ago

I solved the problem. In Server.xml at i set path to chain.pem instead fullchain.pem. The problem only on iOS.

DScrp commented 2 years ago

But there was a very strange problem with UDP. Now UDP works fine at desktop Safari and Firefox (MacOS), Firefox (Win), and mobile Android, iOS. But in Google Chrome and Edge UDP works only with VPN. But if switch transport to TCP, then everything works without VPN. Checked on computers with Windows and Mac. I can't understand what could be causing this problem. In javascript player debug i see events "connectionstatechange connecting" and after a few seconds "connectionstatechange disconnected". But as soon as I turn on the VPN and reload page, "connectionstatechange connecting" change to "connected". Or if i set TcpForce to "true". This problem only in Chrome and browsers on its engine.

Do you have any guesses as to what this might be? I am install OME on two servers, everywhere the result is the same. With UDP and Chrome not connecting, with VPN and UDP (or TCP or other browsers) - works fine. Sometimes it happens that it connects via UDP in Chrome (without VPN). But very, very rarely.

Logs:

[2022-03-21 21:11:23.307] I [SPRtcSig-T3334:2870496] Signalling | rtc_signalling_server.cpp:232  | New client is connected: <ClientSocket: 0x7f59f0001130, #34, Connected, TCP, Nonblocking, 111.222.333.444:7046>
[2022-03-21 21:11:23.641] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(1)/Stream total(1)/App total(1)
[2022-03-21 21:11:23.645] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7060 / Packet type : 0 GateType : 0
[2022-03-21 21:11:34.612] W [SPICE-U10000:2870500] ICE | ice_port.cpp:745  | Mismatched ufrag: P06a (ufrag in peer SDP: Y8uF)
[2022-03-21 21:11:34.612] I [SPICE-U10000:2870500] ICE | ice_port.cpp:783  | Update the client to the port list: to 111.222.333.444:7061 from 111.222.333.444:7060
[2022-03-21 21:11:34.613] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(2)/Stream total(2)/App total(2)
[2022-03-21 21:11:34.676] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7061 / Packet type : 0 GateType : 0
[2022-03-21 21:11:45.603] W [SPICE-U10000:2870500] ICE | ice_port.cpp:745  | Mismatched ufrag: bRtB (ufrag in peer SDP: P06a)
[2022-03-21 21:11:45.603] I [SPICE-U10000:2870500] ICE | ice_port.cpp:783  | Update the client to the port list: to 111.222.333.444:7062 from 111.222.333.444:7061
[2022-03-21 21:11:45.603] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(3)/Stream total(3)/App total(3)
[2022-03-21 21:11:45.669] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7062 / Packet type : 0 GateType : 0
[2022-03-21 21:11:56.600] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(4)/Stream total(4)/App total(4)
[2022-03-21 21:11:56.605] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7072 / Packet type : 0 GateType : 0
[2022-03-21 21:12:07.613] W [SPICE-U10000:2870500] ICE | ice_port.cpp:745  | Mismatched ufrag: 44/A (ufrag in peer SDP: T0bl)
[2022-03-21 21:12:07.613] I [SPICE-U10000:2870500] ICE | ice_port.cpp:783  | Update the client to the port list: to 111.222.333.444:7052 from 111.222.333.444:7072
[2022-03-21 21:12:07.614] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(5)/Stream total(5)/App total(5)
[2022-03-21 21:12:07.672] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7052 / Packet type : 0 GateType : 0
[2022-03-21 21:12:18.611] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(6)/Stream total(6)/App total(6)
[2022-03-21 21:12:18.613] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7050 / Packet type : 0 GateType : 0
[2022-03-21 21:12:28.732] W [SPICE-U10000:2870500] ICE | ice_port.cpp:745  | Mismatched ufrag: qYAe (ufrag in peer SDP: kNtL)
[2022-03-21 21:12:28.732] I [SPICE-U10000:2870500] ICE | ice_port.cpp:783  | Update the client to the port list: to 111.222.333.444:7054 from 111.222.333.444:7050
[2022-03-21 21:12:28.736] I [SPRtcSig-T3334:2870496] Monitor | stream_metrics.cpp:114  | A new session has started playing #my.site#camera-12/channel-3 on the WebRTC publisher. WebRTC(7)/Stream total(7)/App total(7)
[2022-03-21 21:12:28.794] I [SPICE-U10000:2870500] ICE | ice_port.cpp:779  | Add the client to the port list: 111.222.333.444:7054 / Packet type : 0 GateType : 0

Maybe the error is related to warning "Mismatched ufrag"? How can i fix this?

getroot commented 2 years ago

Try explicitly setting the server's network interface by setting the IP address instead of * in and of .

<Publishers>
  <WebRTC>
    <IceCandidates>
        <TcpRelay>xxx.xxx.xxx.xxx:3478</TcpRelay>
        <TcpForce>true</TcpForce>
        <IceCandidate>xxx.xxx.xxx.xxx:10000/udp</IceCandidate>
        <TcpRelayWorkerCount>1</TcpRelayWorkerCount>
    </IceCandidates>
  </WebRTC>
<Publishers>
DScrp commented 2 years ago

I have tried these settings. Does not work.

getroot commented 2 years ago

Hmm well, it looks like there's something unexpected about your network. It only works when you are connected to a VPN, so you should check the firewall or NAT on your PC when VPN is not connected. The server doesn't seem to be the problem. I'm running OME in various environments, but I've never experienced anything like that. I just checked that UDP plays fine in chrome too.

It seems like a good attempt to create an AWS EC2 instance and test it in a public IP.

DScrp commented 2 years ago

There are no firewalls. I tried on three completely different computers with different operating systems: Win10 and MacOS. And the problem is only in Chrome browser. Desktop Firefox, Safari and mobile browsers works fine.

getroot commented 2 years ago

image

Would you like to check the websocket message(signaling) in your chrome like I captured? Is Answer SDP sent multiple times? Looking at the log you uploaded, after "New client is connected", "Add the client to the port list" should be printed only once, but it is strange that it is printed continuously. Answer SDP seems to be sent multiple times. Are those logs from testing on demo.ovenplayer.com?

DScrp commented 2 years ago

I use my player based on OvenPlayer. If there was no connection within 10 seconds, then it connects again. Because of this, there are multiple connections in the logs. Now I have disabled this and the player connects only 1 time.

I check websocket message in Chrome:

2022-03-23_02-17-52

  1. {"command":"request_offer"}

  2. {candidates: [{candidate: "candidate:0 1 UDP 50 xxx.xxx.xxx.xxx 10003 typ host", sdpMLineIndex: 0}] code: 200 command: "offer" id: 1221238068 peer_id: 0 sdp: "v=0\r\no=OvenMediaEngine 103 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE bEm5Mw Oa0oXx\r\na=group:LS bEm5Mw Oa0oXx\r\na=msid-semantic:WMS lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED\r\na=fingerprint:sha-256 CE:D3:C6:AB:F9:F8:2F:19:DC:BD:D1:CD:DC:8D:34:11:F1:CC:48:48:FA:85:BC:9C:94:FD:9C:4F:5F:62:76:04\r\na=ice-options:trickle\r\na=ice-ufrag:QrNWfl\r\na=ice-pwd:Z1juGsrDXfPiFkobze9MB8xt4VUnNvwl\r\nm=video 9 UDP/TLS/RTP/SAVPF 100\r\nc=IN IP4 0.0.0.0\r\na=sendonly\r\na=mid:bEm5Mw\r\na=setup:actpass\r\na=rtcp-mux\r\na=msid:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED rqXyG2OsC9IpS83V74AR1MgtblPo05KjNhex\r\na=extmap:1 urn:ietf:params:rtp-hdrext:framemarking\r\na=rtpmap:100 H264/90000\r\na=fmtp:100 packetization-mode=1;profile-level-id=42e01f;level-asymmetry-allowed=1\r\na=ssrc:4091287708 cname:KYCDcTprWsqPIkfu\r\na=ssrc:4091287708 msid:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED rqXyG2OsC9IpS83V74AR1MgtblPo05KjNhex\r\na=ssrc:4091287708 mslabel:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED\r\na=ssrc:4091287708 label:rqXyG2OsC9IpS83V74AR1MgtblPo05KjNhex\r\nm=audio 9 UDP/TLS/RTP/SAVPF 101\r\nc=IN IP4 0.0.0.0\r\na=sendonly\r\na=mid:Oa0oXx\r\na=setup:actpass\r\na=rtcp-mux\r\na=msid:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED OzspXHE05TjVYhxaMUcoB6Gnilu4kJAQDCLF\r\na=rtpmap:101 OPUS/48000/2\r\na=fmtp:101 sprop-stereo=1;stereo=1;minptime=10;useinbandfec=1\r\na=ssrc:1354674223 cname:KYCDcTprWsqPIkfu\r\na=ssrc:1354674223 msid:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED OzspXHE05TjVYhxaMUcoB6Gnilu4kJAQDCLF\r\na=ssrc:1354674223 mslabel:lrTzoutnUq1CSeBHGMvk8bAh7JmNfYj04ZED\r\na=ssrc:1354674223 label:OzspXHE05TjVYhxaMUcoB6Gnilu4kJAQDCLF\r\n" type: "offer"

  3. command: "answer" id: 1221238068 peer_id: 0 sdp: "v=0\r\no=- 894814847246516550 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE bEm5Mw Oa0oXx\r\na=msid-semantic: WMS\r\nm=video 9 UDP/TLS/RTP/SAVPF 100\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:ym+L\r\na=ice-pwd:24qISBPMJSRQTAq1wE190CKG\r\na=ice-options:trickle\r\na=fingerprint:sha-256 F4:5F:18:C4:38:6E:5C:79:95:DF:3C:FE:F4:4E:12:77:6A:AB:2D:D6:8A:4D:AC:4F:57:84:45:54:19:16:CA:22\r\na=setup:active\r\na=mid:bEm5Mw\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:100 H264/90000\r\na=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\nm=audio 9 UDP/TLS/RTP/SAVPF 101\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:ym+L\r\na=ice-pwd:24qISBPMJSRQTAq1wE190CKG\r\na=ice-options:trickle\r\na=fingerprint:sha-256 F4:5F:18:C4:38:6E:5C:79:95:DF:3C:FE:F4:4E:12:77:6A:AB:2D:D6:8A:4D:AC:4F:57:84:45:54:19:16:CA:22\r\na=setup:active\r\na=mid:Oa0oXx\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:101 OPUS/48000/2\r\na=fmtp:101 minptime=10;useinbandfec=1\r\n" type: "answer"

  4. candidates: [{candidate: "candidate:3028854479 1 udp 2113937151 4477cb12-c843-4a96-90a3-361df27c5781.local 49542 typ host generation 0 ufrag ym+L network-cost 999" sdpMLineIndex: 0 sdpMid: "bEm5Mw" }] command: "candidate" id: 1221238068 peer_id: 0

getroot commented 2 years ago

Both my development environment and the commercial environment I run play fine with UDP. I tried to reproduce your problem, but I couldn't. Does it reproduce the same in https://demo.ovenplayer.com in your environment?

image

If you find the cause of the problem, please share it for all of us.

DScrp commented 2 years ago

Yes, I've tried at https://demo.ovenplayer.com/ UDP works fine at desktop Safari and Firefox (MacOS), Firefox (Win), and mobile Android, iOS. But in Google Chrome and Edge UDP works only with VPN. But if switch transport to TCP, then everything works without VPN.

getroot commented 2 years ago

Hmm... Could it be an IPv6 related issue?

I was able to find an article like the one below. https://groups.google.com/g/discuss-webrtc/c/R3RMOblxdP8?pli=1

Would you like to try turning off IPv6 on Local PC once? I am not using IPv6 in my environment.

DScrp commented 2 years ago

This is very similar to our case. Do you mean my local computer with Win10? I check my ip on site https://whatismyipaddress.com/ IPv6: Not detected. In Win10 i turn off Ipv6 too. But nothing has changed. Sometimes it works in Chrome, but very rarely. If you reload the page 10-20 times, it will connect in Chrome without VPN. Could this be related to my ISP?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.