search

ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.29k stars 632 forks source link

WebRTC is not smooth in 2.2.1 version #2868

Closed golgetahir closed 3 years ago

golgetahir commented 3 years ago

Short description

Customer uses 1.9.1 in an old hardware and wants to switch 2.2.1 on a new hardware Sending the stream with RTMP to both, in webRTC the old version is performing very good but the new version have frame dropping issue. This is being tried by several transcoded videos and with the command;

ffmpeg -thread_queue_size 2048 -f decklink -video_input sdi -audio_input embedded -i 'DeckLink Mini Recorder 4K' -threads 4 \
-filter_complex "[0:v]scale=-2:1080,fps=25,split=outputs=2[1080a][1080b]" -map "[1080a]" -map "[1080b]" -map 0:a \
-c:v libx264 -tune zerolatency -c:a aac -b:a 128k -preset veryfast -b:v 3000k -maxrate 3000k -bufsize 15000k -crf 25 -pix_fmt yuv420p -g 50 -f tee \
"[select=\'v:0,a\':f=flv:onfail=ignore]rtmp://10.0.0.222/WebRTCAppEE/hq-rlt-studio-5-1| \
 [select=\'v:1,a\':f=flv:onfail=ignore]rtmp://172.16.241.106/WebRTCAppEE/hq-rlt-studio-5-1"

The screen record of the problem; https://drive.google.com/file/d/1I-r_wxjFkZfxIKoLoSIJ8aITVAixik-4/view

The server sits inside an internal network so it is not reachable for us.

Environment

Steps to reproduce

  1. Send the RTMP stream with the given ffmpeg command
  2. Watch the stream from any browser
  3. Notice if there is frame drops.

Expected behavior

Video should play smoothly.

Actual behavior

There are dropped frames in the playback.

Logs

Place logs on pastebin or elsewhere and put links here

Join Google Group and be a part of Ant Media Server Community.

zaleschoenborn commented 3 years ago

We are seeing very similar issues and wondering if there's any updates

zaleschoenborn commented 3 years ago

Key points: 1 - stutter is most noticeable with very slow video (see our color bars mpg, included as a link below. this works very well to reproduce issue due do the limited block of motion at the bottom) 2 - stuttering only happens with H.264 (VP8 has the same issue but it manifests as more of a periodic blur than stutter) 3 - only happens on highest quality stream in adaptive settings 4 - happens no matter how you publish (RTMP or web rtc) 5 - There is no setting we've found to make it better so far

Here's a link to the colorbar video that will produce the stuttering every time reliably to test against.

SelimEmre commented 3 years ago

Hi All, I tried this issue both of 1.9 and 2.2.1 versions in my test environment. I couldn't reproduce this issue. Here are our reproduced steps:

  1. We created 2 Instances in OVH
  2. We installed v1.9.1 Ant media Server on one of the created Instances and v2.2.1 version on the other.
  3. We broadcast a stream to both with the same FFmpeg command
  4. Our test team(4 people) played both of them

As a result of the testing, we unanimously decided that v2.2.1 has better video quality.

zaleschoenborn commented 3 years ago

Please watch the moving part of the color bars And you'll see the stuttering.

We did find a workaround which was to increase the number of threads in the encoding

SelimEmre commented 3 years ago

Hi @zaleschoenborn,

Thanks for your clarification. Let me give some details about this issue. I have tested colorbars.mpg file that you provided. I couldn't see any stuttering. Is there anything did I miss?

OBS Dashboard: https://imgur.com/a/huLkUvI OBS Settings: https://imgur.com/a/JCmWQrB and https://imgur.com/a/CIitwtt

WebRTC Statistics: https://imgur.com/a/pEfFVIa Ant Media Server configs: https://imgur.com/a/4mHGTSZ

Default stream mp4 record: https://drive.google.com/file/d/11Mr5vi2Kc-oI1_Im7OfBCHSE31U-TRUi/view?usp=sharing Adaptive stream mp4 record: https://drive.google.com/file/d/1YrRZnqgKM_LzvmkoFKjTPRqdnGGiATCh/view?usp=sharing

belthesar commented 3 years ago

Hi folks!

I'm seeing this as well with a bog-standard vanilla install of Ant Media Server Enterprise via Docker. I'm testing purely via localhost for right now, ingesting via RTMP, using the default LiveApp for output. I'm running with a clock on stream, and I'm only getting video updates every 1 second (to the millisecond). Audio is smooth.

Looking at logs, I'm seeing that the "current video bitrate" is is lower than the client target bitrate, but being new to Ant, I'm unsure if that's anything to write home about. Tossing log snippet below: 

2021-03-15 21:11:19,755 [vert.x-eventloop-thread-6] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1992655750 for stream 471695652251280002360086 current video bitrate: 4300526 audio bitrate: 96000 webrtc client target bitrate: 4884000         
2021-03-15 21:11:21,126 [vert.x-eventloop-thread-5] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1360981287 for stream 471695652251280002360086 current video bitrate: 1463493 audio bitrate: 96000 webrtc client target bitrate: 5068000         
2021-03-15 21:11:24,760 [vert.x-eventloop-thread-6] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1992655750 for stream 471695652251280002360086 current video bitrate: 3826308 audio bitrate: 96000 webrtc client target bitrate: 4898000         
2021-03-15 21:11:24,768 [network_thread - 1414] org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:GetPlayoutUnderrunCount                                                                                                                       
2021-03-15 21:11:25,910 [vert.x-worker-thread-11] io.antmedia.enterprise.adaptive.StreamAdaptor - Stream adaptor queue size: 0 for stream: 471695652251280002360086                                                                                       
2021-03-15 21:11:26,123 [vert.x-eventloop-thread-5] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1360981287 for stream 471695652251280002360086 current video bitrate: 1839840 audio bitrate: 96000 webrtc client target bitrate: 5068000         
2021-03-15 21:11:29,756 [vert.x-eventloop-thread-6] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1992655750 for stream 471695652251280002360086 current video bitrate: 3740869 audio bitrate: 96000 webrtc client target bitrate: 4898000         
2021-03-15 21:11:31,123 [vert.x-eventloop-thread-5] io.antmedia.enterprise.webrtc.WebRTCAdaptor - Client:1360981287 for stream 471695652251280002360086 current video bitrate: 2092040 audio bitrate: 96000 webrtc client target bitrate: 5068000         
2021-03-15 21:11:31,135 [network_thread - 1400] org.webrtc.Logging - Tag:WebRtcAudioTrackExternal - Message:GetPlayoutUnderrunCount                                                                                                                       
2021-03-15 21:11:32,092 [vert.x-worker-thread-11] io.antmedia.enterprise.webrtc.codec.VirtualVideoEncoder - frame id: 2582247960 capture TimeNs: 578532000000

Edit: Looking at chrome://webrtc-internals - looks like the culprit is a huge slough of dropped frames: https://i.imgur.com/JZHqMob.png