search

CasparCG / server

CasparCG Server is a Windows and Linux software used to play out professional graphics, audio and video to multiple outputs. It has been in 24/7 broadcast production since 2006. Ready-to-use downloads are available under the Releases tab https://casparcg.com.
GNU General Public License v3.0
911 stars 268 forks source link

CasparCG Server V 2.3.3 LTS Stable Audio out of Sync #1380

Open Rotti96 opened 3 years ago

Rotti96 commented 3 years ago

Expected behaviour

When I start playback a video with embedded audio the audio seems to be out of sync from the beginning. The audio seems to be delayed, so thats why I can't work with Audio Delay. May you can help? Are there other people wo have this problem? Tried with former Versions of CasparCG Server. Only in V2.0.7 the Playback seems to be right displayed.

Environment

Screenshots

If applicable, add screenshots as complementary information.

sendust commented 3 years ago

We got the opposite result. CCG 2.3.0 ; audio and video are exactly in sync CCG 2.3.2 ; audio eariler than video 30ms CCG 2.3.3 ; audio and video are exactly in sync

Tested with Window 10 CCG 2.3.2 CCG 2.3.3 Decklink Embedded output (11.5.1) HD 59.94i Setup

CCG_AVSYNC

premultiply commented 3 years ago

Screen-Consumer and System-Audio can never be a reference as they are full asynchronous to everything. It is worse as they are independent to each other and to multiple outputs. They do not share a common reference clock.

If you do such tests please make sure that you use a professional output device which is connected and genlocked to a reference clock that even provides the time reference to a proper NTP daemon on the system.

Rotti96 commented 3 years ago

Okay, thank you for your Feedback. We create just an environment for some tests for new interactive graphics to play with caspar. I just wonder because 2.0.7 is allways in sync. We only take focus on the graphics so that they work as they should. Thank you guys.

pbelbin commented 3 years ago

Screen-Consumer and System-Audio can never be a reference as they are full asynchronous to everything. It is worse as they are independent to each other and to multiple outputs. They do not share a common reference clock.

If you do such tests please make sure that you use a professional output device which is connected and genlocked to a reference clock that even provides the time reference to a proper NTP daemon on the system.

Hmmmm. @premultiply : You seem to be saying that PCs just can't achieve good audo/video sync using anything other than 'pro' interfaces. Perhaps not 'perfectly synchronized', but, sufficiently good for most cases seems to be managed by most of the stand-alone apps for video playback, so I think of this as more of a CasparCG implementation issue rather than 'it can never be done' kind of problem.

So, the question is: how can CasparCG be improved to the point of having parity with other PC playback software?

premultiply commented 3 years ago

Yes, correct. PCs are not build to provide proper synced video and audio output without additional hardware support. You can try to keep them in sync with the standard output devices by maintaining some small buffers but that always introduces frame dropping/doubling and/or audio resampling/streching.

pbelbin commented 3 years ago

I can see how sending audio and video via different devices, that have no correlation to each-other would be difficult to keep in sync. eg: mainboard audio and discrete video card output. I suspect the way this situation is supposed to be handled is that a device is nominated as the 'clock source' and playout devices (both audio and video) are supposed to subscribe to the clock heartbeat event happening so that they both react within a very short period of time, and there should be 'pacing' of the content being streamed out so that it's kept reasonably well synchronized / low jitter.

If both the audio and video are being sent out via a single displayPort connection, via a discrete video card (eg: NVIDIA P1000), you'd think that the clock source on-card would be able to be used for both audio and video, and that this would allow both to be kept in sync more tightly, and prevent 'drift over time', which I've seen quite a bit of. ie: audio and video playout seem well synchronized initially, but, after a couple of hours, there is noticeable discrepancy between audio and video delivery.

Interestingly, I've been making some small changes to allow the output audio device to be selectable. The interesting thing here though, is that, the way by which the list of audio devices is known and accessed seems to be via an interface that aggregates all audio devices, regardless of what hardware it's on. I kind of wonder if there is an issue regarding audio sync with video because of the use of this abstract interface mechanism? Perhaps it's a bit 'too abstract' to allow the audio and video to stay in sync with video. It's total speculation at this point. It could also be that the abstract mechanism offers a way to synchronize, but we're just not using it currently.

As I mentioned earlier: PC stand-alone playback software seems to manage to solve this issue. I don't see why CasparCG should not also be able to solve the issue.

stevespaw commented 3 years ago

Sorry as just an onlooker to this conversation, Why not use NDI instead of the screen consumers? The audio should be locked there. Just curious......

Julusian commented 3 years ago

From reading this I am getting a little confused as to whether this is about sync or drift.
You are right in expecting there to not be any drift, that should not be happening. But casparcg does not guarantee that different consumers will output their data in sync, even if they are both going out via the same port on a graphics card. This is the benefit of the decklink consumer, as the audio and video stay together all the way through to the card.

For the drift, I know rather little about the current openal implementation as most users I see are either using decklinks or are running in cases where the drift wont be noticed, or can be corrected by restarts overnight. I am pretty sure that casparcg is simply passing audio samples into a library, and assuming that library is draining at the correct rate. It is very possible that for a lot of framerates it is being fed the wrong number of samples for each frame. Or maybe it is a bug in the version of openal we are using, which looks to be from 2014. I suspect this library is not the best for what we need either, as we are doing only some very simply audio output

pbelbin commented 3 years ago

Sorry as just an onlooker to this conversation, Why not use NDI instead of the screen consumers? The audio should be locked there. Just curious......

NDI output would be fine if the receiving switcher can ingest it. If you're playing out video+audio to something that has physical inputs (HDMI/SDI/whatever), then you have to output the audio/video via physical outputs. Horses for courses.

pbelbin commented 3 years ago

From reading this I am getting a little confused as to whether this is about sync or drift. You are right in expecting there to not be any drift, that should not be happening. But casparcg does not guarantee that different consumers will output their data in sync, even if they are both going out via the same port on a graphics card. This is the benefit of the decklink consumer, as the audio and video stay together all the way through to the card.

For the drift, I know rather little about the current openal implementation as most users I see are either using decklinks or are running in cases where the drift wont be noticed, or can be corrected by restarts overnight. I am pretty sure that casparcg is simply passing audio samples into a library, and assuming that library is draining at the correct rate. It is very possible that for a lot of framerates it is being fed the wrong number of samples for each frame. Or maybe it is a bug in the version of openal we are using, which looks to be from 2014. I suspect this library is not the best for what we need either, as we are doing only some very simply audio output

I think both drift and sync are concerns.

In this case, I define 'drift' as the progressive increase in lack of synchronization between audio and video being output.

How closely synchronized the audio and video are initially, I can't say, my 'casual observation' is that it's 'close enough', but, definitely, if you leave it overnight, you will find that there is significant offset between audio and video (my experience, anyway), and it's unusable.

Restarting CasparCG resets the audio & video to being in sync again, but, it seems pretty clear that this is also the starting point, and that the drift (progressive loss of sync between audio and video) is well underway (again). Personally, I am observing the drift happening to the extent that CasparCG has to be restarted way more often than every 24 hours. Even after 2 hour of video playout, it needs to be restarted.

That people using CasparCG seem to take it as 'normal' that it has to be restarted every 24 hours seems very strange to me. It sounds a bit like there are some issues that need to be found and fixed.

Using a more recent library could well help. Perhaps looking more at how other playback applications manage to keep audio and video delivery in sync will help shed some light on how to make it reliable?

dimitry-ishenko commented 3 years ago

The fact that 2.0.7 is staying in sync (at least according to @Rotti96) means that it is possible.