Audio and Video go out of sync when using video-sync=display-resample

[meta3ark]

Important Information

• mpv v0.37.0-787-gab504514
• Windows 10 Pro
• Shinchiro
• Change of behaviour between 22/10/2023 and 29/10/2023 shinchiro sourceforge builds
• Nvidia GTX 1060 6GB, 551.86

Reproduction steps

Play a local 23.976Hz mkv file on a frame rate matched TV for about 10 minutes.

mpv --no-config --vo=gpu-next --video-sync=display-resample

Expected behavior

Audio/Video does not go out of sync after a period of time

Actual behavior

After approximately 7 minutes I observed mis-timed frames in stats and the audio (lip-sync) goes noticably out of sync with the video.

Log file

video-sync-display-resample-log.txt video-sync-audio-log.txt

Sample files

Any 23.976hz mkv file

Additional Info

using Revert "d3d11: calc vsync interval on real stats... fixes the problem related issue #13952 and possibly #13439

[kasper93]

using Revert "d3d11: calc vsync interval on real stats... fixes the problem

Ok the master now. Reopen if you see other issues.

[meta3ark]

Tested and working great. Many thanks.

[meta3ark]

I thought it was fixed but it is still happening but now every approx 20 minutes. A very noticable video stutter/jerk. Stats -> 1 delayed frame. It resumes and video/audio do appear to be in sync, so better than originally.

Issue does not occur after reverting to October 2023 build and changing nothing else.

[Jules-A]

I wasn't getting this before but it looks like I've been seeing this (very slightly) in the past few weeks though it could be something else at fault.

EDIT: Good: https://github.com/zhongfly/mpv-winbuild/releases/tag/2024-06-14-a41cdf0 Bad: https://github.com/zhongfly/mpv-winbuild/releases/tag/2024-06-15-6c90836

[meta3ark]

I'll test build a41cdf0 tonight.

4 commits between those builds https://github.com/mpv-player/mpv/commit/6c908363cd17cf14738b005d59668d1400cb2510 https://github.com/mpv-player/mpv/commit/ab0a50874bccbf1e11cc15346e87c78ef50acae3 https://github.com/mpv-player/mpv/commit/3fa09458daf94e1a463b9eb6bd0271476c5b4263 https://github.com/mpv-player/mpv/commit/daa3264d3f81c28c368ce8e791ca4d2d370dc726

[kasper93]

I thought it was fixed but it is still happening but now every approx 20 minutes. A very noticable video stutter/jerk. Stats -> 1 delayed frame. It resumes and video/audio do appear to be in sync, so better than originally.

Issue does not occur after reverting to October 2023 build and changing nothing else.

Find the regression range, we will not search for bugs that only you can reproduce and not reliably at best.

[meta3ark]

I thought it was fixed but it is still happening but now every approx 20 minutes. A very noticable video stutter/jerk. Stats -> 1 delayed frame. It resumes and video/audio do appear to be in sync, so better than originally. Issue does not occur after reverting to October 2023 build and changing nothing else.

Find the regression range, we will not search for bugs that only you can reproduce and not reliably at best.

Not sure why you ignored comment from Jules-A https://github.com/mpv-player/mpv/issues/14141#issuecomment-2198031170 who can also reproduce it.

Last night I tested the version before your original fix and immediately after. It was fixed. I then tested versions suggested by Jules and can confirm one of the 4 commits listed is the cause. It takes around 20 minutes to stutter and happened on 23.976 and 25/50 fps content.

https://github.com/mpv-player/mpv/commit/6c908363cd17cf14738b005d59668d1400cb2510 https://github.com/mpv-player/mpv/commit/ab0a50874bccbf1e11cc15346e87c78ef50acae3 https://github.com/mpv-player/mpv/commit/3fa09458daf94e1a463b9eb6bd0271476c5b4263 https://github.com/mpv-player/mpv/commit/daa3264d3f81c28c368ce8e791ca4d2d370dc726

[kasper93]

So you confirm that this build is good and this is bad

[Jules-A]

So you confirm that this build is good and this is bad

For me they both had issues still, however the one you marked as good is (slightly) better when seeking (at least according to the stats, didn't notice anything myself).

EDIT: I tested the CI build of https://github.com/mpv-player/mpv/actions/runs/9507590076 and I'm still seeing it so maybe it's some external library rather the MPV commits? EDIT2: going back a few more days it's fine (mpv commits didn't seem like they could affect video playback).

[kasper93]

@meta3ark: Can you provide regression range for the issue you are seeing?

@Jules-A: Could you report own issue, with all required info, as your problem is not related to the changes discussed here.

Thanks.

[meta3ark]

I tried good/bad and latest github action build. All working without delayed frames/stutters. Libmpv builds from 15/06 onwards have stutters/delays. Nothing obvious I can see and the 20 min plus testing periods make finding a reproducible scenario difficult. Closing this as it's a different issue i think.

[Jules-A]

I tried good/bad and latest github action build. All working without delayed frames/stutters. Nothing obvious I can see and the 20 min plus testing periods make finding a reproducible scenario difficult. Closing this as it's a different issue i think.

Hmm... That is odd, afaik the "good" build was between those 4 commits you listed and the "bad" build was before them? (I think labeling was wrong way around). Also yeah, it's not easy to test and possibly is another issue but I don't really have time to investigate it more and report myself right now. (I suspected it wasn't actually the MPV commits at fault and if that's the case is it really an MPV issue in the first place?). Even without the dropped/delayed frames randomly occurring, the normal Vsync jitter is doubled for me in the newer builds though it's still so low that alone shouldn't be causing any issues.

[kasper93]

I tried good/bad and latest github action build. All working without delayed frames/stutters.

Latest, you mean for current master branch?

Libmpv builds from 15/06 onwards have stutters/delays.

By libmpv build you mean the build from shinchiro?

If our CI build works fine, it is likely not a mpv issue. Nor it is ffmpeg or libplacebo one, because we build them too from master branches. So the issue if I understand correctly is likely somewhere else. Let us know if you find something.

Since the regression is pretty new, you have build for each commit on our CI, if you click on green tick mark. Also shinchiro and zhongfly build are there, so it should be possible to find the regression range.

[Jules-A]

Since the regression is pretty new, you have build for each commit on our CI, if you click on green tick mark. Also shinchiro and zhongfly build are there, so it should be possible to find the regression range.

According to their replies they had the same regression range as me which I posted 3 days ago

Good: https://github.com/zhongfly/mpv-winbuild/releases/tag/2024-06-14-a41cdf0 Bad: https://github.com/zhongfly/mpv-winbuild/releases/tag/2024-06-15-6c90836

I then tested versions suggested by Jules and can confirm one of the 4 commits listed is the cause.

While there were 4 MPV commits between those builds, the CI build right before the commits doesn't actually have the same libraries due to the time of builds.

[meta3ark]

Every build of mpv is working as it should regardless of source of the build. Shinchiro, zhongfly or CI.

libmpv is not available from CI i think, so that was both shinchiro and zhongfly. 14/06 worked, 15/06 onwards stutters. I looked at their commits for that day, vapoursynth change only. seems very unlikely to be a problem since i don't use (directly) vapoursynth. An upstream lib change could be the cause, but that's a needle in the haystack...

I'll try to follow the VS2022 instructions to create a master/CI-a-like libmpv build and test that. It would be nice if it was part of the CI builds by default.

[meta3ark]

It appears to be happening with a local build of libmpv-2.dll. I'll build a dll per commit from the 14/06 to 15/06 and see if i can figure out when the problem started.

[kasper93]

It would be nice if it was part of the CI builds by default.

It is quite annoying to configure shared library build, with static libraries dependencies in Meson, that's why it is not available. And likely this will not change until this is implemented on Meson side https://github.com/mesonbuild/meson/pull/13086#discussion_r1576818094

[meta3ark]

It is quite annoying to configure shared library build, with static libraries dependencies in Meson, that's why it is not available. And likely this will not change until this is implemented on Meson side mesonbuild/meson#13086 <(comment)

no worries, as a c# dev msbuild does 99.999% of build stuff for me.

Rightly or wrongly i'm doing two passes

.\ci\build-win32.ps1

then

meson configure build -Dlibmpv=true -Ddefault_library=shared ninja -C build mpv-2.dll

I've added ability to switch between lib versions from ui with an automatic app restart as https://github.com/mpv-player/mpv/issues/11638 prevents me from just closing/disposing libmpv handle and creating a new one.

Testing with 25/50hz for 30 mins plus, didn't error at all for any version. I continue to test...

[meta3ark]

Tested local build of e07470a against same release build from shinchiro. No delays/stutters with local build, but had some with the shinchiro build. Problem appears to be downstream, so this issue can stay closed.

[kasper93]

but had some with the shinchiro build

Would be really nice if you could track down the issue, with shinchiro build. I think a lot of people are affected/using those ones.

[meta3ark]

Would be really nice if you could track down the issue, with shinchiro build. I think a lot of people are affected/using those ones.

At some point i'll try them again, I've spent 20 hours plus on this and no one else has reported anything in last few weeks.

Also, I don't know what the benefit's of the 3rd party builds (hard to grok and twice the size) are now that i can quickly compile any commit directly from mpv source on windows.

[kasper93]

Sure, no worries. If it works for you it is fine. Thanks for trying to figure it out in the first place.

[kasper93]

I double checked the difference between 2024-06-14-a41cdf0 and 2024-06-15-6c90836 builds, including 3rdpary libs versions. I don't really see anything that would explain dropping frames. There are mostly small changes, nothing significant. And especially if local build done with build-win32.ps1 doesn't show the issue, there must be something strange going on. Obviously build-win32.ps1 doesn't include all the things that zhongfly build does, but still I don't see how the playback could be affected. Unless it is some strange libass or decoding issue. Really don't know, hard to pin-point anything.

[meta3ark]

Doubt it helps in the tracing, but it was only for libmpv-2.dll. mpv.exe (shin, zhon or local) were all working OK.

Testing requires firing up my projector (finite and very expensive lamps) with rest of equipment that consumes much energy/cost. Therefore, I only test when i'm already using it. A couple of dual monitor mpv FPS estimation bugs makes my attempts to test on my dual monitor desktop impossible.

[Jules-A]

including 3rdpary libs versions

Is there an easy way to see all the commits from all 3rd party libs between a specific time period?

[kasper93]

Is there an easy way to see all the commits from all 3rd party libs between a specific time period?

Go to build details, and click on Packages Version. I don't see anything of importance and if those changes really regress the stability of the playback, we need to diagnose what is really going on. I don't know if you @Jules-A have easier way to reproduce the issue, but you could try doing custom builds with reverted 3rdparty libs. We need to narrow down it step by step, no other way, I think.

[Jules-A]

Go to build details, and click on Packages Version.

Ah I was hoping for something easier, it's still hard to check that way and unless I'm blind or not understanding the changes, the only thing I can see that might affect things would be https://github.com/libass/libass/pull/780 ? While I was testing on a file that was using ass subs, I still see almost doubled Vsync jitter when playing a tx3g sub video between those builds that may be related or at least is really strange. I did rule out the Vulkan changes since it occurs in DX11 though I'm not sure if it's used for the shaders?

I don't really have time to set up building / reverting stuff manually sorry. I may be mistaken but I think the drops when running on my usual config were when it was caching (CPU use is increased quite a bit when it occurs). Maybe increasing cache sizes can help reproduce, otherwise maybe using a computer with a slower CPU?

[kasper93]

Ah I was hoping for something easier, it's still hard to check that way and unless I'm blind or not understanding the changes, the only thing I can see that might affect things would be https://github.com/libass/libass/pull/780 ?

It is quite literally impossible for this change to regress performance or make any significant change in normal use of libass. I agree it looks suspicious in this regression range, but I cannot see how it can create a problem. Also @meta3ark confirmed that his local build is not affected, while it also includes this change you are asking about. Also CI builds are not affected.

Do you see a difference between v3 and "normal" build?

[meta3ark]

Local build and app using libmpv restarted after resuming pc from sleep. I saw 1 stutter (1 delayed frame reported) about 5 mins into playing an ota hdhomerun (ts) local network stream. Nothing reported for next 3 hours, mix of local file and streams, all over 30 mins long.

I don't have any ass sub content. I can't test v3 as htpc i5 is too old.

[kasper93]

@zhongfly: would you be possible for you to rerun this build without libass version change? Just for the sake of eliminating this factor. I don't follow how to build those builds with GH actions.

[zhongfly]

@zhongfly: would you be possible for you to rerun this build without libass version change?

Unfortunately, it can't. Even building the exact same version again is difficult. This is because each run is based on the latest cache and updates all git repositories before starting building.

If necessary, I can build one where all dependencies are up to date, but reset libass to libass@306bf76. https://github.com/zhongfly/mpv-winbuild/actions/runs/9859120502

ps:difference between 2024-06-14-a41cdf0 and 2024-06-15-6c90836 https://www.diffchecker.com/BL2k5rwK/

[kasper93]

If necessary, I can build one where all dependencies are up to date, but reset libass to libass@306bf76. https://github.com/zhongfly/mpv-winbuild/actions/runs/9859120502

I guess that works, thanks. @Jules-A could you try this build?

[meta3ark]

I guess that works, thanks. @Jules-A could you try this build?

i'll try it later too

[Jules-A]

I guess that works, thanks. @Jules-A could you try this build?

Thanks zhongfly, I will test this tonight.

Do you see a difference between v3 and "normal" build?

I didn't end up testing the non-v3 builds again enough to tell (just ~15mins but I didn't see any issues in that time). Originally I did test it when it first occurred and it didn't make a diff then though.

EDIT: Tested for at least 3hrs and didn't see any issues.... (at least when it's in foreground and I don't perform any seeks) Swapping between subtitles caused quite a bit more dropped/mistimed frames than master though (not that it's an issue since it's rarely done, just thought it was interesting). I'm not going to bother mentioning the a/v drift/vsync drift diff since it's like 10x lower with https://github.com/mpv-player/mpv/pull/14058.

[meta3ark]

Non-v3 zhongfly libmpv build only

3 x 30 mins (3 viewings) of 23.976 fps content. No delayed frames 60 mins of mixed fps yt streams. No delayed 4 x 30 mins of hdhomerun network 25 fps streams. 1 or 2 delayed frames per viewing at random times with noticable stutters.

EDIT: hdhomerun stream: 1 delayed frame occured a few minutes after seeking back 5 seconds, so mpv back buffer in use.

[Jules-A]

Okay, I also tested master for another 2hrs30mins and wasn't able to reproduce... Now I'm questioning if it was even the program itself rather than some background process I was unaware of (I also noticed this time I didn't have the mouse mouse randomly reappearing for the whole time I was testing). Only thing I did differently this time was update Windows 10 to latest monthly update and delete both AMD's and MPV's cache. Maybe whatever it was got fixed already? I really don't know anymore. There's still a lot of dropped/mistimed frames when MPV isn't in foreground but pretty sure that's a separate issue so I think I might call it quit testing and wait on some other synchronization fixes to land... Sorry guys that I wasn't able to be of more help.

[meta3ark]

I have app, cpu and gpu monitoring during playback at all times (being doing this for years at this point on same hardware). I see all sorts of CPU spikes, rarely, if ever, causes problems. When i have the time i will try to take this monitoring to the next level, such as timestamping delayed frames.

I updated Win10 in the last week too (ShutUp10 blocks for months at a time) so maybe M$ and their no testing policy is to blame. It's a real worry that Win10 is almost EOL. Win11 is terrible and resource hungry. Talking of resource hungry, stats.lua causes a 10% cpu usage spike on display (first page), but i don't really use it and doesn't cause drops, so minor issue for me.

Since i still have some dropped frames i need to do some more investigation but i'm also not at all sure if mpv is at fault now. I'll try zhongfly latest master tonight.

[meta3ark]

Tested libmpv from https://github.com/shinchiro/mpv-winbuild-cmake/releases/tag/20240712 for a few days and I haven't seen any noticable stutters. When I've spot checked stats, no delayed frames reported for any content. So, I think we are good and I can now implement https://github.com/mpv-player/mpv/issues/11638, thanks @kasper93.