search

bluenviron / mediamtx

Ready-to-use SRT / WebRTC / RTSP / RTMP / LL-HLS media server and media proxy that allows to read, publish, proxy, record and playback video and audio streams.
MIT License
11.64k stars 1.47k forks source link

OBS Studio 30.0.0 / WHIP to MediaMTX 1.2.1: x264 encoder leads to WebRTC/WHEP stutters, NVIDIA NVEC encoder causes packet errors #2600

Closed rse closed 9 months ago

rse commented 10 months ago

Which version are you using?

v1.2.1

Which operating system are you using?

Describe the issue

I'm trying to get OBS Studio 30.0.0's new WHIP ingest to work with the latest MediaMTX 1.2.1 and a 1080p60 stream. When I use the OBS Studio (software) encoder "x264" I can smoothly play the video on MediaMTX with HLS egress, but on WebRTC/WHEP egress the video stutters. When I switch to the OBS Studio (hardware) encoder "NVIDIA NVEC" MediaMTX on WHIP ingest the HLS parts reports H.264 packet problems:

2023/10/29 07:56:26 INF [HLS] [muxer test] is converting into HLS, 2 tracks (H264, Opus)
2023/10/29 07:56:27 INF [HLS] [muxer test] destroyed: muxer error: unable to extract DTS: SPS not received yet yet

The WebRTC/WHEP egress does not show any errors, but also does not work when using the "NVIDIA NVEC" encoder.

Describe how to replicate the issue

Use OBS Studio 30.0.0 and WHIP ingest with either "x264" or "NVIDIA NVEC" encoders and try a MediaMTX egress with WebRTC/WHEP. The NVIDIA NVEC parameters in OBS Studio are:

But I tried dozens of other NVIDIA NVEC parameters and it always leads to the same DTS/SPS-errors inside MediaMTX. So, doesn't seem to be related to the particular parameter combination above. When ingesting with RTMP (and exactly the same NVIDIA NVEC parameters!) it works smoothly, but obviously then results in a higher overall latency (the reason I wanted to switch to WHIP).

I also found out that when using the "x264" software encoder the stuttering is actually just on the WebRTC player(!) side. When I in parallel receive the stream via SRT the video looks good enough. So, the NVIDIA NVEC encoder totally breaks the ingesting into MediaMTX and the x264 encoder lets at least the WebRTC playing to stutter. 1

Did you attach the server logs?

no

Did you attach a network dump?

no

rse commented 10 months ago

It's interesting, the MediaMTX WebRTC playback heavily depends on the particular OBS Studio encoder and its parameters. The NVEC encoder breaks the situations entriely. The "x264" encoder with some parameter variants lets the player not even start displaying the stream, others cause stream stutters. But at least one combination resulted in a smooth MediaMTX WebRTC playback with the "x264" encoder:

And this is with both WHIP and SRT ingests into MediaMTX. So, MediaMTX really has not an issue on the WHIP or SRT ingest side. But it has an issue on the WebRTC/WHEP side as it looks. At least just the above combination leaded to a smooth player experience until now. I also figured out that for me any Preset higher than "Fast" or any Tune different to "Low Latency" breaks the player experience.

rse commented 10 months ago

Ok, I've done even more investigation with OBS Studio 30.0.0 and MediaMTX 1.2.1 snapshot as of 2023-11-13. When you compare the attached video you see in the middle a browsersource of an analog clock (see https://github.com/rse/analogclock if you want to try yourself). The program is ingested into MediaMTX via SRT. On the top-right is a HLS egress/playback of MediaMTX and on the left bottom is a WebRTC egress/playback of MediaMTX. Observe the sweep hand of the original clock in the middle and the HLS playback one: they are both move totally smooth. But on the left bottom one can clearly see that the WebRTC playback stutters. AND NOW IT COMES: This observed 1s jumps are coming from the H.264 keyframe interval. If I increase this to 2s, the sweep hand jumps in 2s frequency, if I increase this to 3s, the sweep hand jumps in 3s frequency. So, the WebRTC playout seems to playback just the keyframes!?

https://github.com/bluenviron/mediamtx/assets/221273/c75c0312-07f0-4747-afbd-148239d5cc58

rse commented 10 months ago

Ok, I've tested even more parameter combinations for "x264" inside OBS Studio 30.0.0 in order to get smooth both WebRTC and HLS playout through MediaMTX. The only combination which really worked is:

So, my final conclusion: it seems like this isn't really an issue with MediaMTX, but the WebRTC playout even inside a browser like Chrome is very very sensible to the used encoder paramters. Slight changes immediately breaks the WebRTC playout.

rse commented 10 months ago

With the above mentioned parameters OBS Studio 30.0.0 is finally sending out a stream to MediaMTX via SRT and receive it back via WebRTC and HLS. The stream renders smoothly and there is even an amazing low overall latency. Check the screenshot here:

Screenshot 2023-11-13-13-40-14

The overall roundtrip (OBS Studio to MediaMTX in the cloud any back again) for SRT+WebRTC (center to left-bottom) is just 1,0s and for SRT+LL-HLS (center to right-top) it is just 2.0s.

leoluk commented 10 months ago

What are your SRT parameters? When I publish to mediamtx using SRT, the delay gradually increases over time until I restart OBS. Are you seeing this too?

rse commented 10 months ago

No, latency remained more or less constant for me in my last tests, at least with a duration of 30-60 minutes only. I've no special parameters to SRT ingest except latency=120000. And the above x264 encoder parameters, of course.

nerdbaggy commented 9 months ago

I was able to get the WHIP working with these settings Rate Control - ABR (Everything was always jittery) Bitrate - 8000 kbps Keyframe Interval - 0 CPU Usage - superfast Profile - None Tune - fastdecode