search

Yalir / sfeMovie

sfeMovie is a simple C++ library that lets you play movies in SFML based applications. It relies on FFmpeg to read medias and remains consistent with SFML's naming conventions.
http://sfemovie.yalir.org/
GNU Lesser General Public License v2.1
114 stars 37 forks source link

Fix audio desynchronization when audio gets late #3

Closed Ceylo closed 8 years ago

Ceylo commented 13 years ago

For now, when video is late, images are skipped to re-synchronize. But there's no such method for audio when it gets late.

Ceylo commented 12 years ago

As audio can no more be late at start (except if the decoder is buggy), this issue should be much less noticeable.

bluekirby0 commented 10 years ago

Generally a good container format will provide some information that helps maintain synchronization between audio and video streams. You can use this information (when available) to hold the current video frame until the audio catches up.

Actually, I assumed ffmpeg would handle this particular aspect. Is there a specific scenario that can duplicate this issue so I can get a better idea of what is going on?

Ceylo commented 10 years ago

Actually the issue is mainly with SFML audio buffering and audio driver delays, data on FFmpeg's side is fine, it usually have good timestamps.

SFML preloads approximatively 3s of audio data, and the initial delay with the audio driver (ie. between data when has finished being sent to SFML and went it starts playing) can go up to 1s. What is being done for now is that when starting playing, sfeMovie waits for OpenAL to indicate that audio playback did start (ie. playing position > 0). This is the best way I've found to at least start playback synchronized: once we know audio is playing, video starts and can catch up audio by playing the first video frames faster.

While playing, if audio gets late at some point, you have two possible solutions: (1) make video wait (2) skip some audio samples

I was rather for the idea (2) in order to keep the playback on time, the main issue being that if you start skipping samples, you can no more rely on the playing offset given by OpenAL, and if you update the offset given to OpenAL, it'll flush preloaded data and will start loading again 3 secs of data (but remember, you're already reading 3 secs in advance on the FFmpeg's audio stream, so you have to seek backward if you don't want to miss data, which isn't supported by sfeMovie yet).

Solution (1) is probably easier but not perfect too: I can't check every video frame update if video should wait for audio to catch up because the playing offset returned by OpenAL (and thus SFML) is not regularly updated enough (it gets updated every 0.2s if I remember well). So sfeMovie may think audio is late just because the playing offset wasn't updated yet.

bluekirby0 commented 10 years ago

I'm not familiar with OpenAL much...(I know what it is but have not poked into its internals and API). Does it provide a callback function that you can use to help maintain synchronization perhaps? Something that provides the current positional data and is called only when that data is updated?

Ceylo commented 10 years ago

I don't know... but even if it does, it would require some support on SFML's side, because sfeMovie doesn't use OpenAL directly, it only uses SFML's API.

I looked at how SFML uses OpenAL: it queries for the AL_SEC_OFFSET property on a OpenAL audio source, and this information is not updated very often. That's what other users seem to have reported too and in some way it look normal because it depends on what the audio hardware sends back.

Ceylo commented 8 years ago

Closing as it does not appear to be noticed by users, so it means it happens too rarely to be annoying :)