Video/audio out of sync on some platforms

[anigart]

Hi,

When playing a video in a Windows VM running on a Mac, everything is fine and video and audio are in sync. On the same machine, not in a VM (so on native MacOS), audio is about one second ahead, breaking lipsync. This also happens on other native Windows machines I tried. The only difference is that the machines where it does not work have a lot more computing power than the VM where it works. Could it be that modern machines introduce a sync issue due to their speed?

I added some debug statements and audio.play() is called less than 10ms before the first frame renders, so it seems they start correctly at about the same moment. I suspect video is playing too slow then and audio runs ahead (audio does sound normal, not too fast).

Is this a known issue?

Thanks for any help.

[anigart]

Quick update on this issue: this is still a problem with the latest code and natives (although from my perception it seems the severity is reduced compared to an older version). Audio and video are out of sync, breaking lip sync noticeably. My source video (webm format) plays nicely synced using VLC.

[SimonIT]

The natives didn't change since your first test. I made some optimizations from the java side. Do you updated your .jar's? If not, it could be that the problem is fixed because it was from the java side

[anigart]

I used the latest java code. I have no way of measuring it but it seems improved, though it's still very noticeable. Thanks for resuming work on this.

[SimonIT]

Okay, I will make some tests. Maybe I will find the source of the offset

[roiniti]

I see that this problem its still unfixed, can i suggest to add some function to sync the audio, like set the audio in order to start 200 miliseconds after the video starts something like this that i aded to CommonVideoPlayerDesktop: (i made it myself and i don't if it works 100% fine) `

private boolean audioStarted = false;
private boolean videoStarted = false;

private long async = 0;

public void setAudioAsync (long as) {
    async = as;
}

@Override
public boolean update () {
    if (decoder != null && !paused && playing) {
        if (startTime == 0) {
            // Since startTime is 0, this means that we should now display the first frame of the video, and set the
            // time.
            startTime = System.currentTimeMillis();
        }
        //Starts later on positive async and instant on negative or 0 async
        if (!audioStarted && audio != null && System.currentTimeMillis() >= startTime + async) {
            audioStarted = true;
            audio.play();
        }

        //Starts later on negative async and instant on positive or 0 async
        if (videoStarted || System.currentTimeMillis() >= startTime - async) {
            videoStarted = true;
            boolean newFrame = false;
            if (!showAlreadyDecodedFrame) {
                ByteBuffer videoData = decoder.nextVideoFrame();
                if (videoData != null) {
                    if (texture == null) texture = new Texture(currentVideoWidth, currentVideoHeight, Format.RGB888);
                    texture.bind();
                    Gdx.gl.glTexImage2D(GL20.GL_TEXTURE_2D, 0, GL20.GL_RGB, currentVideoWidth, currentVideoHeight, 0, GL20.GL_RGB,
                        GL20.GL_UNSIGNED_BYTE, videoData);
                    newFrame = true;
                } else {
                    playing = false;
                    if (completionListener != null) {
                        completionListener.onCompletionListener(currentFile);
                    }
                    return false;
                }
            }

            showAlreadyDecodedFrame = false;
            long currentFrameTimestamp = (long)(decoder.getCurrentFrameTimestamp() * 1000);
            long currentVideoTime = (System.currentTimeMillis() - startTime);
            int difference = (int)(currentFrameTimestamp - currentVideoTime);
            if (difference > 20) {
                // Difference is more than a frame, draw this one twice
                showAlreadyDecodedFrame = true;
            }
            return newFrame;
        }
        return false;
    }
    return false;
}`

also i added in play this: audioStarted = false; videoStarted = false;