Smooth playback of `video-sync=display-*` at FPS below the monitor's maximum

[hooke007]

Currently set video-sync=display-resample would make mpv always try to match the monitor's highest refresh rate. For those 240~360Hz monitors, enabling this option would introduce much frame-drop/delay.

Expected behavior of the wanted feature

Add the new option like video-sync-target-fps=xxx to avoid this trouble.

Alternative behavior of the wanted feature

N/A

Log file

N/A

[christoph-heinrich]

The increased overhead comes from interpolation and not from video-sync=display-*, but reducing the targeted frame rate would certainly help with resource usage.

Instead of targeting a specific refresh rate, it would make more sense to cap it to a max so that it can still line up with lower refresh rates in mixed refresh rate systems.

But even that isn't an optimal solution, because when e.g. a cap of 80fps is set and the window is on a 120hz monitor, then targeting 60fps instead of capping out on 80fps would probably result in smoother playback.

Now with that being said, is there even a need for interpolation on such monitors? I'd imagine that the refresh rate is high enough that slightly imperfectly timed frames would be hardly noticeable. I have never used such a monitor though, so it's hard to say.

[hooke007]

The increased overhead comes from interpolation

I didn't enable interpolation with display-resample. Enabling interpolation would increase more GPU consumption.

But even that isn't an optimal solution, because when e.g. a cap of 80fps is set and the window is on a 120hz monitor, then targeting 60fps instead of capping out on 80fps would probably result in smoother playback.

This could ref some games, they could use fixed 1/2 or 1/4 refreash rate as vsync target.

[Dudemanguy]

Shouldn't --override-display-fps work for this?

[hooke007]

Shouldn't --override-display-fps work for this?

Unfortunately, it would make more delayed/dropped frames.

override-display-fps=120

[Dudemanguy]

Can you post a log with that option set to 120 really quick?

[hooke007]

fps.log (start with override-display-fps=120) fps2.log (apply override-display-fps=120 on the fly)

[Dudemanguy]

[ 0.104][v][vo/gpu] Assuming 120.000000 FPS for display sync.

So yes, the target FPS is actually configurable and a quick look at the code confirms this. However, when using a display-* video sync mode, this leads to worse results. The reason is because those modes are timed directly to vsync. Since your monitor has such a high refresh rate, the vsync block returns twice as fast (i.e. at a 240hz rate instead of 120hz) which proceeds to mess up mpv's timing code which is being told that the display FPS is 120. So the feature you want technically exists (override-display-fps), but it's at odds with how display-* is designed so it can't possibly work well. I'll alter the title a bit for clarity for clarity.

As for implementing something like this, it wouldn't be impossible but it would take some work and careful thinking.

[WhoMI7]

I have somewhat similar issue.

Currently set video-sync=display-resample would make mpv always try to match the monitor's highest refresh rate.

In my case Display Fps (specified) is monitor's current refresh rate and when using video-sync=display-* the Display Fps (estimated) is double of specified Display Fps and with mistimed and delayed frames.

Issue only persists with gpu-api=d3d11, vulkan and opengl doesn't show any drops and delays.

[Dudemanguy]

That sounds like a bug.

[WhoMI7]

Maybe it is. I found 2 fixes for these, (Nvidia GPU)

1. Turning off G-sync/freesync. (Not reliable)
2. Setting Monitor Technology to Fixed Refresh Rate for mpv in Nvidia Control Pannel. (Con: Black screen for 2 secs whenever mpv is opened, closed and fullscreen in and out.)

Can't reproduce my issue on AMD gpu.

[hooke007]

G-sync is incompatiable with mpv. It's another known issue.

[WhoMI7]

Yeah, maybe I should open another issue for my problem, plus I have some questions too.

Edit: Fixed my issue, it was a driver bug in my case. Maybe yours too? https://www.guru3d.com/files-details/display-driver-uninstaller-download.html

[dontpokethebear3893]

+1 for this feature request. I am currently toying with smooth motion and refresh rate handling on my 175hz monitor with gsync and this would solve most of my problems.

[snylonue]

Now with that being said, is there even a need for interpolation on such monitors? I'd imagine that the refresh rate is high enough that slightly imperfectly timed frames would be hardly noticeable. I have never used such a monitor though, so it's hard to say. Actually, the differences between interpolation on and off are noticeable on a 240Hz monitor.

Is this issue related to the drawback described in wiki?

Finally, --video-sync=display-* currently comes with one important drawback: Due to OpenGL's rather severe limitations when it comes to timing, the only way to reliably figure out when vsyncs happen is to actually draw a frame on every vsync. The consequence of this is that, even for 24 Hz video, you need to draw frames at 60 Hz even if they are the same frame over and over again - thus increasing power usage by a factor of 2x-3x in such a case. This is usually somewhat alleviated by caching the frame result, and redrawing the cached frame, instead of scaling and rendering the source frame every time. The problem that there's less time to render subtitles on a 60 Hz screen remains.

[hooke007]

@Dudemanguy Could you consider adding it into https://github.com/mpv-player/mpv/milestone/3 ?

Since your monitor has such a high refresh rate, the vsync block returns twice as fast (i.e. at a 240hz rate instead of 120hz) which proceeds to mess up mpv's timing code which is being told that the display FPS is 120.

Unfortunately, I noticed obvious frame-drops in my current display(4k95hz). I cannot even smoothly playback a 720p24fps short video. This sounds really ridiculous. Which means large numbers of modern monitor would suffer this pain.

[Dudemanguy]

A framerate limiter is a good feature in general but out of scope for the upcoming release. We can revisit this after 0.37 I think.

[kasper93]

A framerate limiter is a good feature in general but out of scope for the upcoming release. We can revisit this after 0.37 I think.

While this can be added, the limiter is directly contradict the premise of display-sync. The main problem is that we cannot schedule vsyncs accurate enough in software to make perfectly smooth playback, without any strutter. That's why display-sync exist to rely on implicit vsync to sync frames correctly.

So actually the frame limiter will not improve situation, because it will introduce stutter in itself. The proper way to use display-sync is to set your display to certain refresh rate and let mpv sync to that.

Of course frame limiter is viable option, but like I said it will not do what you want it to do in fact.

[Jules-A]

While this can be added, the limiter is directly contradict the premise of display-sync. The main problem is that we cannot schedule vsyncs accurate enough in software to make perfectly smooth playback, without any strutter. That's why display-sync exist to rely on implicit vsync to sync frames correctly.

That's not entirely true, with Freesync it's possible but Freesync only works properly in fullscreen. Currently you need to use a vf to multiply the video's FPS though which isn't really ideal for some since it means you can't use interpolation.

[ghost]

Creating a custom modeline is probably the more elegant and less headache-inducing solution. If you have a custom modeline, it's already debatable whether or not you actually need display-resample to get even pull-down.

The only non-elegant part is that switching between modelines on Windows is just annoying and scripting it requires a lot of external freeware tools that I hate keeping on my system. But modeline switching is probably beyond the scope of mpv since it would need to be OS-specific.

[bsolar17]

I think I'm encountering a similar issue on MacOS on a MacBook Pro with 120Hz monitor. The issue is very apparent when using Vulkan, but not with libmpv.

With this command, a 4K video is pretty jerky, showing a lot of Mistimed/Delayed frames happening during playback:

mpv --no-config -vo=gpu-next -gpu-context=macvk -fs -video-sync=display-resample test.mp4

With this command the same video is smooth:

mpv --no-config -vo=gpu-next -gpu-context=macvk -fs test.mp4

And with this one the video is also smooth, with only a handful of Mistimed/Delayed frames happening at the very beginning:

mpv --no-config -vo=libmpv -video-sync=display-resample -fs test.mp4

[Akemi]

@bsolar17 try with different options of --macos-render-timer, eg precise and system.

[bsolar17]

@Akemi, I tried with both options but they did not improve the jerkiness.

[ghost]

G-sync is incompatiable with mpv. It's another known issue.

Is there any update on this? Also where is the issue about it

[fideliochan]

It has been 1 year, updates ?

[na-na-hi]

This could ref some games, they could use fixed 1/2 or 1/4 refreash rate as vsync target.

This is already possible with --opengl-swapinterval/--d3d11-sync-interval and --display-fps-override. You need to manually calculate the display fps after being divided by the swap interval, but this works for me.

[snylonue]

This could ref some games, they could use fixed 1/2 or 1/4 refreash rate as vsync target.

This is already possible with --opengl-swapinterval/--d3d11-sync-interval and --display-fps-override. You need to manually calculate the display fps after being divided by the swap interval, but this works for me.

but such options won't solve the frame dropping issue as pointed out before.

[na-na-hi]

but such options won't solve the frame dropping issue as pointed out before.

I don't think setting a larger swap interval was mentioned before in this issue. That is required for this to work.

You need to set the interval to a value larger than 1. If the video driver is working properly, the "estimated" display fps should be reduced by that factor (if not, it will still be the real display fps no matter what the value of --display-fps-override is).

For example, after specifying a factor of 2, the "estimated" display fps should be around 120 for a 240 Hz display. This option throttles the rendering speed so mpv thinks that the display has a lower fps. Then set the value of --display-fps-override to match that estimated fps. This way mpv also thinks that the display's refresh rate matches the rendering speed, so it detects nothing abnormal and will render at the reduced framerate.