intersink/intersrc - A/V delay

[hjanuschka]

having a simple input file running in a loop, and producing a TCP output. produces - over time, like 12h+ a significant A/V delay.

it is not really about the time/hours, it seems once the cpu usage on the host is peaking/blocking the x264 encoder, audio continues flawless but video lags slightly behind.

folks at voc2mix, also have suffered that issue, they somehow overcome it with restructuring their pipelines. https://github.com/voc/voctomix/issues/58

however, they cut out the tcp input - for that to work, and don't really support dynamic inputs (in general they drift more in a SDI-only direction)

currently i am playing around with: https://github.com/RidgeRun/gst-interpipe

you have any idea how to solve the problem (overscaling CPU is one thing, but its no guarantee that even the biggest cpu is working 100% of the running time) - restarting (stop/play) the mixer/output solves it but interrupts the service.

[moschopsuk]

Hello @hjanuschka

55 Covers some of this as well.

Are you able to share any of the aspects of the file and system you are running it on ?

We have a version that now uses interpipe to solve some of the issue of intersink/intersrc due to the way they split buffers and create a new clock. Sadly I've not had time to backport these changes. But we have had much more success with them.

[moschopsuk]

As for the CPU problem you could run brave with one of the GST_DEBUG option running to identify if there are buffers that are get stuck.

https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c

[hjanuschka]

debugging didnt bring any light, it show sometimes dropped frames, sometimes high latency.

sometimes, frames are missing in the V pipeline, and sometimes they are stuck for a couple of ms and then run in double-speed. its just too unreliable.

i'v been trying to hlssink the whole thing, with a 3-5s delay, and kept restarting the tcp/mixer on a regular basis :o - but that feels way too hacky.

any chances i can give the interpipe branch?

thx for your feedback.

details: (does not happen on small files with low resolution)

resolution: 1920x1080 sample file: https://vzaar.com/videos/1570191

when testing, with TCP output, it sometimes is "just" client issue, like vlc is lagging. using "local" gl output, or direkt decklink sink. removes alot of cpu usage, but still introduces lag over time, so the main problem must be the inter* elements.

[hjanuschka]

system: 8 cores, nvidia card (although no nvdec is used and stuff like that) - using nvdec, might reduce the issue a bit, since it might free a couple of cpu resouces, although it won't solve the problem :/ running: native on ubuntu, also running on k8s (tried with cpu limits)

[moschopsuk]

any chances i can give the interpipe branch?

I can't promise an ETA but when i can find some time to work on this I'll push some changes. Out internal version of this is more progressed but since it ties into more BBC systems it a little hard to backport.

system: 8 cores, nvidia card (although no nvdec is used and stuff like that) - using nvdec, might reduce the issue a bit, since it might free a couple of cpu resouces, although it won't solve the problem :/

Brave uses playbin3 for files, this elements combines many smaller elements together to achieve de-muxing and decoding of the streams in the file. Should you have HW acceleration and the correct libraries installed it should use.

[hjanuschka]

@moschopsuk really would appreciate, to see how it can be ported to interpipes, let me know when there is something, i can look at, would also love to contribute to it.

ohh, yeah, thats correct about playbin3, anyway, cannot get rid of the occasianal stutters and resulting delays.

[SonnyWalkman]

There's discussion on replacing intersrc and intersink with Ridge Runs interpipe? Is therre any update here? Not sure if interpipe has been approved to be included in gstreamer mainstream as yet. Seems awfully long-time to get it including as a usable element? I feel interpipe is favourable here..