fzwoch
commented
2 years ago I feel like I have seen the same with the Teleport plugin. I believe it just any source that is prone to buffer starvation. I believe that OBS has to buffer something to maintain smooth playback and maintain audio video sync. And that's fine. obs_source_set_async_unbuffered may be way to disable this buffer and will see the unsmoothed result.
But I suspect that on a discontinuity/buffer starvation case OBS will try to resync the source to a new base 0 time. E.g. due to netwrok contraints you may end up with 500 ms of no data but then will get all the data from the 500 ms. OBS in the meantime detected a discontinuity, and will base the next sample that 500 ms late as the new base for the source. That will result in the latency you are seeing as now there are 500 ms of data being build up in the buffer.
Instead of treating the delayed sample as a new base one would either have to increase playback rate or drop data to catch up with the old time base. But is all speculation on my side. I never looked at these internals on the OBS side, and I believe you should ask them about how these things actually work. I just see the plugin API and try to make most sensible things out of it - but unfortunately it is inconclusive on some details.
normen
rodrigograca31
TL;DR: It seems theres a buildup of OBS and Gstreamer buffers for camera inputs, theres a possible workaround in PR #66
Introduction First let me explain the issue I am experiencing, it seems to be quite common in OBS under certain circumstances. There is a creeping latency for source inputs, especially for network based cameras. After a few hours of streaming I get up to seconds of latency on the video streams. This issue can be seen across all kinds of OBS plugins in github issues and I was experiencing it with both the OBS-NDI and OBS-Gstreamer plugins.
The good news is I think I found a fix for this plugin (see PR #66) - the bad news is I don't completely understand what the actual issue is. At least for my setup NDI cameras now run in low latency sync for hours in OBS with this fix.
Issue Description I have two different systems, each with 3 cameras connected to OBS, everything running at 30fps. One is a beefy computer at work with fancy NDI cameras (~3%CPU load) and one less powerful at home with RTSP based cameras (~40%CPU load). The beefy NDI computer manages to get around the issue for longer while with the less powerful one it happens pretty quickly. First a few frames, then more and more. Audio is captured separately in these setups so the delay is noticeable in comparison to the audio.
Investigation/Workaround Basically there seems to be two buffers that can start to fill for some reason. First theres the source input buffer in OBS which can be disabled with
obs_source_set_async_unbuffered(as done in the NDI and Decklink plugins for example). Then theres theappsinkbuffer in gstreamer which can already be disabled via the "Block..." options in obs-gstreamer. Still, to get completely rid of any adding up of latency I had to tell the appsink to drop frames when the sink isn't fast enough withgst_app_sink_set_drop.This seems to work for NDI cameras that come in via the gst-plugin-ndi gstreamer plugin. Just today I recorded a whole concert without rising latency issues. For the RTSP cameras I still get some kind of buildup of latency but with these settings it doesn't go to seconds anymore, rather hundreds of milliseconds, which is still bad for lip sync though.
Interestingly, when using the OBS-NDI plugin and disabling OBS source buffering (
obs_source_set_async_unbuffered) I can avoid the rising latency but I get jittery video input instead, so something seems to go awry with frame delivery/pickup. Seems like the NDI side of things doesn't have an output buffer like gstreamers appsink and instead chooses to chop up frames instead?Conclusions Now, I don't completely get what this means. It seems that OBS is simply unable to pick up all the samples and either increases its own buffers or blocks, causing the plugin to increase buffers. Is it actually only picking up samples at the output framerate? In that case the issue should be less obvious for people that run e.g. the cameras at 30fps and OBS at 60fps I suppose? At any instance, as said theres multiple plugins where this issue comes up so maybe it's more of a bug or badly documented OBS API.
Not expecting a solution right away but I thought I put my findings in words and archive it here at least. Thanks so much for the obs-gstreamer plugin, its a blast ❤️