search

ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.24k stars 620 forks source link

RTMP to WebRTC drops *alot* of frames while HLS is fine #2980

Closed k0nr4d closed 3 years ago

k0nr4d commented 3 years ago

Short description

If I broadcast a video from OBS or XSplitter to Antmedia over RTMP, the output can be played perfectly using HLS. from /WebRTCAppEE/play.html?name=konrad&playOrder=hls

However, if I change the playorder to webrtc, it plays terribly. It appears to drop about 90% of the frames or more. Audio also skips, it's absolutely terrible and unwatchable. The output shows zero dropped frames in OBS, play.html (not playere) on antmedia shows dropped frames sometimes but other times it skips despite showing no dropped frames.

I have made a video using my phone of the issue, you can see it here: https://drive.google.com/file/d/1PZTXb35Z-LOa7qCsjacqxVZHfIurwlaP/view?usp=sharing

The video is made using firefox as playback but the issue is consistent on all browsers.

Also interestingly, the RTMP stream does not show up in the list of streams. https://prnt.sc/zym35j Shows 1 live stream However if I go into https://prnt.sc/zym4vu it shows no live streams.

Environment

I have tried this with several versions of antmedia enterprise edition, including the newest on Ubuntu 18.04. Video was taken using Enterprise Edition 2.2.1 20201029_2042 Issue occurs at the very least also on Enterprise Edition 2.1.0-SNAPSHOT 20200609_0637 Enterprise Edition 2.2.0-SNAPSHOT 20200721_1150 Screenshot of WebRTCAppEE settings: https://prnt.sc/zymnle This happens with Adaptive disabled as well. The server does not really have any load right now, there are no other streams in Antmedia and effectively zero http traffic headed to it.

Steps to reproduce

  1. Broadcast from OBS to RTMP Endpoint (https://prnt.sc/zymbuo). Issue also exists when using CBR.
  2. Attempt to view stream as WebRTC
k0nr4d commented 3 years ago

Sorry just to mention also, just for the sake of experimentation I tried using XSplit instead of OBS and the issue persisted.

mekya commented 3 years ago

Hey @k0nr4d ,

Thank you for letting us know. It seems that this problem is about tuning the OBS or Vmix.

Could you make sure that you have the correct settings in OBS? https://github.com/ant-media/Ant-Media-Server/wiki/Open-Broadcaster-Software-Publishing#tune-for-ultra-low-latency-streaming

we have also docs for VMix and Xsplit

Please let us know whether it works for you

k0nr4d commented 3 years ago

Hello, Yes, I found and followed that guide initially. I don't think the issue is with the ingress. If it was, then the HLS version wouldn't be playing properly. I wouldn't get good HLS playback without good RTMP input. I am however getting (very) poor WebRTC playback.

k0nr4d commented 3 years ago

Even setting the RTMP bitrate very low like 500 or 1000 kbit doesn't resolve the WebRTC output issue. HLS consistently plays fine although sometimes has some "flickering" as seems to be described in #2637. This flickering interestingly does not occur on the WebRTC playback.

mekya commented 3 years ago

Hey @k0nr4d

HLS and WebRTC Playback have some differences. I mean you can play a stream with HLS perfectly but you cannot with WebRTC. Because WebRTC is optimized for real-time streaming. Having B frames may cause problem in WebRTC playback.

Maybe we can make a quick demo together with you and invite other guys from the community to join the call?

k0nr4d commented 3 years ago

@mekya If you could, I would be grateful. I tried disabling b-frames in OBS but the result is the same.

mekya commented 3 years ago

Hey @k0nr4d

Can we make it on Sunday at 4:00 PM GMT+3 and can we let other guys from the community join the video call?

k0nr4d commented 3 years ago

@mekya Sounds good to me and yes no problem letting other's join.

danielbiagi commented 3 years ago

I experienced the same with my AWS setup with Vmix input, was wondering if 2.3 would fix that.

Can you try 2.3 also?

k0nr4d commented 3 years ago

I experienced the same with my AWS setup with Vmix input, was wondering if 2.3 would fix that.

Can you try 2.3 also?

I have no way to try 2.3.0 RC easily unfortunately - all the servers we have running it are in production, except one and that's on a managed server I don't have root access to.

mekya commented 3 years ago

Hi @k0nr4d ,

I think it's a kind of configuration problem. Because I know many projects are using 2.2.1 with RTMP ingesting and WebRTC playback. We can analyze your case and make some demonstration tomorrow.

Here is the event

It's on 4:00 PM GMT+3 tomorrow and google meet link is here -> https://meet.google.com/fnr-aeiw-pcw

k0nr4d commented 3 years ago

Hey, The call is no longer needed on our end, one of my staff figured out the issue. That being said, there may be a bug that needs looking into here none-the-less, or at the very least our finding should be mentioned on the OBS settings page in the documentation. This might be a bit of an edge-case scenario because presumably not everyone messes with the FPS settings.

@mekya The issue was actually that we were broadcasting at 30fps, but we had adaptive streaming in Antmedia enabled, and set lower at 20fps. When we changed it to 30fps as well and with OBS locked at 30fps, it resolved the issue and now it works perfectly. So it seems like the stuttering and frame drops in WebRTC are caused when these settings are mismatched, but HLS manages to handle it. The FPS match also seems to have resolved the intermittent "flickering" issue under HLS described under #2637

@danielbiagi Can you confirm if this resolves the issue on your end as well?

OBS Settings: https://prnt.sc/101ogvd https://prnt.sc/101ogme

mekya commented 3 years ago

Hey @k0nr4d , Thank you for letting me know. Shall we still make this call so that you just explain your find outs in the event? It probably help guys in the community save time.

If you don't wanna do that, no worry it's not a problem.

Regards, A. Oguz

k0nr4d commented 3 years ago

@mekya Yeah no problem, I can hop on the call.

mekya commented 3 years ago

Perfect. See you on tomorrow :)

I've notified the community about this event. https://groups.google.com/g/ant-media-server/c/NKLq-4HJLjM

I'm not sure if anyone joins but we at least meet and talk.

danielbiagi commented 3 years ago

@k0nr4d actually I don't have ABR on in my setup but it doesn't drop as much frames as yours, just a little hiccup every minute or so.

I tried enabling it for a test and synched the FPS of the input with antmedia settings but got some freezes and flickering.

Anyway, hope 2.3 with the new RTMP library fixes it.

@mekya Do you guys have a release date for the AWS Marketplace package?

Thanks

k0nr4d commented 3 years ago

@danielbiagi I installed 2.3.0 RC on two of ours and with a relatively reasonable bitrate it's working well. No flickering, no pixelation, the very rare odd hiccup. It's a bit worse when I first connect to the stream - the first 5-10 seconds has some hiccups and then it clears up. @mekya any ideas on that?

k0nr4d commented 3 years ago

@danielbiagi Try setting your OBS to 20fps under the video settings tab? that is apparently the default used by Ant as mentioned in the group call we had. We also had b-frames set to zero and keyframes set to 1.

mekya commented 3 years ago

@danielbiagi , it seems that we've find out some bugs in 2.3 RC and have fixed them. We plan to release it this week. We'll schedule the update for AWS Marketplace in next week.

@k0nr4d For the first 5-10 seconds, it may try different ice candidate to find the better candidate pair. It may cause this problem.

danielbiagi commented 3 years ago

@k0nr4d I did try it now, the flickering was resolved but there was still some minor freezes and hiccups with adaptive bitrate on.

I found out that Chrome plays with a few hiccups but Firefox is just awful, freezes and hiccups all the time.

Maybe you could try Firefox on Ubuntu 20.04 to see if you also have problems.

Anyways, thanks for the tips ;)

k0nr4d commented 3 years ago

Firefox on Windows10 also performs terribly it seems for me

On Tue., Feb. 23, 2021, 16:04 Daniel Biagi, notifications@github.com wrote:

@k0nr4d https://github.com/k0nr4d I did try it now, the flickering was resolved but there was still some minor freezes and hiccups with adaptive bitrate on.

I found out that Chrome plays with a few hiccups but Firefox is just awful, freezes and hiccups all the time.

Maybe you could try Firefox on Ubuntu 20.04 to see if you also have problems.

Anyways, thanks for the tips ;)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ant-media/Ant-Media-Server/issues/2980#issuecomment-784265964, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACV7CPIP3CSPXEPLXFBHT4TTAO7YJANCNFSM4XZ6432Q .

mekya commented 3 years ago

So the issue is playing streams with adaptive-bitrate in Firefox is awful both on Windows 10 and Ubuntu 20.04.

Do you confirm that? If yes, we can schedule this issue in this week.

k0nr4d commented 3 years ago

@mekya Yes, I confirm. I just tried it now on the freshly installed 2.3.0 updated RC that you sent me (feb 24th build), OBS->WebRTC, 20fps, plays fine on Chrome but skips like crazy on Firefox.

mekya commented 3 years ago

Creating a new issue for this #3014 and closing this one.

rjtoledo commented 3 years ago

@k0nr4d did you manage to try it on firefox android?