Frame doubling (to better support FreeSync/G-Sync)

[pinumbernumber]

mpv version and platform

N/A, Windows 10 x64

Reproduction steps

Play a 24fps video using a monitor which has adaptive sync but which does not have built-in low-framerate compensation. (Most 4K monitors fall under this, for example. A 40-60hz freesync range is typical with no LFC.)

Expected behavior

Each frame should be presented twice, resulting in a 48fps effective framerate and allowing freesync to kick in. The video should therefore play free of typical 24-on-60 judder.

Win10's built-in "Films & TV" player can do this, and so can Edge. Verified using my monitor's OSD. No other player that I've tested can do it. I think it would be a great feature to add for owners of adaptive sync displays.

I would suggest that the video's actual framerate should be multiplied by the largest integer such that the result does not exceed the monitor's reported refresh rate. For example, 24fps content would be doubled to 48 on 60hz displays, and tripled to 72 on 75hz displays. This maximises the likelihood that adaptive sync will kick in.

Actual behavior

The video is presented at 24fps (confirmed with Afterburner's OSD), and freesync does not kick in. The monitor remains at 60fps, and the expected judder is present.

Log file

N/A

Sample files

N/A, any 24fps file

[Akemi]

you can use the fps ffmpeg filter for that, --vf=fps=fps=48. just simply write a script that sets this filter dynamically by observing/reading the display-fps and fps properties, similar to this script.

[haasn]

I thought drivers are already capable of doing this under the hood?

Solving this generally is probably going to be rather tricky, since we don't even know the input framerate. But you could approximate it maybe with --tscale=nearest and some options to control the display FPS? Dunno.

[pinumbernumber]

@Akemi I'll check that out, thanks for the advice. It would be nice to have it built-in instead of a separate lua script, though.

@haasn

I thought drivers are already capable of doing this

Not in general. With FreeSync this is called Low Framerate Compensation. Not all monitors have it, and I don't believe any 4K FreeSync displays do at the moment. Even when it's available, it has been known to cause flicker and other issues. Far better to output somewhere in the native freesync range (usually 40-60).

we don't even know the input framerate

Maybe not in special cases, but the vast majority of files have a fixed known framerate, don't they?

[Akemi]

i don't think this is a feature that will get a native implementation within mpv.

yeah most videos are fixed fps, though not all files provide the correct fps information. mpv calculates an estimated fps but that can only be used after a few frames have been decoded and hence is not available at file start/load.

[Argon-]

Maybe not in special cases, but the vast majority of files have a fixed known framerate, don't they?

Problem is that even when their metadata contains it you don't know if it's correct. I've got plenty of files here claiming slightly wrong values or such that are completely ridiculous (0 or 9000). You don't notice this because players really don't care about this value. Relying on it would definitely lead to problems.

[pinumbernumber]

Well, it looks like my ignorance of video timing techniques is showing. If the framerate isn't reliably known upfront, then the duration of the frame that is about to be presented must at least be known immediately before said presentation, right? So to greatly oversimplify (for example I assume present() returns instantly, which it definitely doesn't), I assume it might look something like this right now:

present(frame.data);
sleep_ms(frame.duration);

The frame doubling would look something like

present(frame.data);
sleep_ms(frame.duration/2);
present(frame.data);
sleep_ms(frame.duration/2);

[haasn]

Actually, couldn't we do it like this? Combining the two things we do know:

1. The display framerate (which for adaptive sync I assume is meant to be an uppter limit)?
2. The presentation duration of the frame (by comparing this and the next frame's timestamp)

We could calculate the number of repetitions as floor(frame_duration / vsync_interval). So for an example of a 41.7ms frame on a 16.6ms monitor, we would end up repeating it twice, each repeat for a duration of exactly 20.85ms, giving us an effective total framerate of 48 Hz (by repeating each 24 Hz frame twice).

[pinumbernumber]

Yes- if I follow you correctly, that would achieve what I'm looking for. The monitor's reported framerate is indeed an upper limit- the lower limit is usually around 40. (Any lower than that requires LFC which is a bit of a minefield and, as I said above, not currently available on 4K monitors).

[grantwwu]

Any possibility we could get this reopened, @Akemi?

[Akemi]

sure.

[zc62]

Just curious, does interpolation do the job?

[xnoreq]

@zc62 The whole point of adaptive sync is NOT to synchronize to the display refresh rate but to just present at a certain rate and the monitor adjusts to it.

The whole implementation should be really simple: 1) the monitor runs at the max refresh rate 2) a configuration option tells mpv the min_adaptive_refresh_rate 3) frame_repeat_count = ceil(min_adaptive_refresh_rate / video_fps) 4) present each frame at 1 / (video_fps * frame_repeat_count) time intervals

Example 1:

• 120 Hz monitor with 40 Hz min refresh rate
• 23.976 fps video
• => frame_repeat_count = 2
• => each frame (including the repeated ones) is presented at ~20.8542 ms intervals (= 47,952 fps)

Example 2:

• same 120 Hz monitor with 40 Hz min refresh rate
• 60 fps video
• => frame_repeat_count = 1
• => each frame (including the repeated ones) is presented at ~16.667 ms intervals (= 60 fps)

Example 3 (worst case):

• awful 90 Hz monitor with 60 Hz min refresh rate
• 50 fps video
• => frame_repeat_count = 2
• => each frame (including the repeated ones) is presented at 10ms intervals (= 100 fps)
• since 100 fps is greater than the monitor max refresh rate there are two simple options (ignoring the more complex vsync case):
  • no vsync => tearing
  • enhanced sync => no tearing, some judder (the higher the monitor refresh rate, the better)

You don't need any interpolation, no waiting for vsync. Just present the frames at a high enough rate (by showing them multiple times if needed) and the monitor will adjust.

I have suggested this years ago. It's way simpler than any of the interpolation/resampling logic implemented right now.

[zc62]

I've recently acquired a freesync display, which has the range set to 57-144Hz. When I play a 24fps video file with mpv in fullscreen, the OSD of my monitor shows something around 72Hz. I think the feature is already there? Nothing needs to be done in mpv. GPU APIs or drivers handle this. You could try to enable the recently added 'Variable Refresh Rate' for d3d11 in win10 1903, but I tried vulkan API, works too.

[magicgoose]

Solving this generally is probably going to be rather tricky, since we don't even know the input framerate.

I think it's okay. It's only needed to know pts of this frame and next frame. And you insert a frame between them (if the interval is not already short enough).
(And if effective FPS is already high enough, then that interval will always be short enough and nothing will be inserted.)

[ghost]

Doubling the framerate is trivial, you can probably construct something with existing libavfilter filters.

[philipl]

FWIW, my freesync display allow frame doubles just fine - and from an nvidia gpu no less.

[Artins90]

FWIW, my freesync display allow frame doubles just fine - and from an nvidia gpu no less.

Could you post your config file? I have a 35-60Hz screen running off a 2080Ti but I can't get freesync to work properly with mpv, for 24 fps videos it uses 2.500 vsync rate resulting in 60 fps instead of doubling to just 2.000 at 48fps.

[philipl]

@Artins90 You're using interpolation. Turn it off.

[ghost]

Do you mean --video-sync=display...?

[philipl]

Sorry, I do mean that - although I think he has to also be turning on interpolation for it to be doing 2.5 rate, right?

[Artins90]

@Artins90 You're using interpolation. Turn it off.

Without interpolation the 24 fps video runs at 60 fps with 2.5 vsync rate. If I delete "video-sync=display-resample" from the config file, the afterburner osd rports 24 fps but my screen can't do 24 fps since it's outside the freesync window and the monitor doesn't support frame doubling on its own.

I would like to know if there is a way to have mpv output each frame of the 24 fps video 2 times, resulting in 48 fps playback, which is within the freesync range supported by the monitor, forcing vsync rate to 2.000 rather than 2.500 would do the trick.

[philipl]

Well, sure, then you're in the same boat as the original reporter. I was just saying my display does frame doubling so everything works fine. There's no config magic that will allow it to work with a non-doubling display.

[ghost]

I don't think interpolation affects how many frames it outputs. It'll just affect performance in various ways.

And also, with --video-sync on default, it won't measure or report any relevant statistics (it'll only show the nominal FPS reported by the OS on the stats screen).

[ghost]

Also I guess you can try --vf=fps=48/1.001 (if I understand the problem correctly, but I didn't try to). Of course this is for testing only and very much not recommended for normal use. It's meant for files with ~23 fps, and won't work 100% correctly (mkv rounded timestamps and all that).

[haasn]

Does this help?

video-sync=display-resample
interpolation=yes
tscale=oversample
override-display-fps=48

That'll make mpv frame double, but I have no idea if that results in usable or sane timing, since it will still be driven by the vsync loop. Actually, maybe you could use that but set it to the maximum fps your display is capable of handling? That would more than frame double though.

[ghost]

This will just set 48 Hz as initial nominal display fps. Then mpv will notice that it can output more and adjust the display fps. (No idea how it works with freesync. I even have a fresync display, but the POS is not recognized as such, and it's probably completely different from the Windows behavior anyway.)

[haasn]

Oh, right, it doesn't change the pattern unless you combine it with --untimed but then the timing is fucked anyway.

[Artins90]

Also I guess you can try --vf=fps=48/1.001 (if I understand the problem correctly, but I didn't try to). Of course this is for testing only and very much not recommended for normal use. It's meant for files with ~23 fps, and won't work 100% correctly (mkv rounded timestamps and all that).

It works, I have never seen video play so smooth, it's quite impressive. OpenGL wouldn't trigger G-sync no matter what, after switching to DX11 and setting: video-sync=audio
vf=fps=48/1.001

now g-sync works when playing the video in fullscreen, windowed mode disables it despite having set g-sync to both windowed and fullscreen in the Nvidia control panel, that's not a problem though.

It would be nice if vf=fps= could be set to 2x the video fps automatically only when the result of such multiplication falls within the variable refresh range of the monitor (in my case 35-60Hz), maybe falling back to "display-resample" automatically when the result of the multiplication is outside the range.

[xnoreq]

Guys, mpv does not support any form of adaptive sync. The monitor refresh rate is whatever you configure in Windows. It might be 60, 75, 120, 144 ... Hz and whatever the video fps, it will be displayed at that fixed refresh rate, even in fullscreen.

Afaik, and I could be wrong here, it also doesn't support "mode setting" depending on video fps, that is change the monitor refresh rate e.g. to 60 Hz when playing a 30 fps video.

[zc62]

I have a similar observation to what philipl had.

My display can do 57-144Hz, and if I enable interpolation and display-resample, the display reports a constant 144Hz.

If I use default interpolation=no and video-sync=audio, a 24fps video would trigger 72Hz for my freesync display (though the actual refresh rate reported by the display OSD fluctuates, don't know why.) The OSD of mpv still reports 144Hz (as wm4 has explained).

[xnoreq]

How does that work? If the desktop refreshes at a constant 144 Hz then that's it for variable refresh non-maximized windows, isn't it?

[xnoreq]

Tried on Windows 10 and does not work. How could it? The DWM would somehow need to adapt to the variable refresh rate of the active window, which could be rendering at 1 Hz and would therefore break the rest of the desktop, other applications.

Fullscreen works, but that has been working for months if not years (contrary to my Dec 2019 statement)

[xnoreq]

Doesn't work for me, then again I'm using an adaptive sync monitor.

So what happens if you have e.g. a 60 fps youtube video running in the browser in the background and you start mpv with fps=48?

Also, what does your monitor actually receive? My monitor shows the (variable) refresh rate in its OSD.

[xnoreq]

Turns out the Nvidia control panel is bugged and messed up my mpv profile some time ago. I had set monitor technology to "use global" setting which is set to "GSYNC compatibility", but nvidia profile inspector showed that it was set explicitly to "Off". Also another related option was set to off.

After cleanup up the profile it works.

The implementation is definitely weird. Using the 72 fps video, my monitor shows 72 Hz, then it rises to 144 Hz where it stays shortly and then drops to 72 Hz again and this repeats over and over again. This is with a YouTube video in the background. Cannot explain the observed behavior, but maybe the driver tries to squeeze in frames from background applications.

[ghost]

If it's VRR, isn't it expected that the framerate changes all the time?

[xnoreq]

Turns out that makes no difference.

In fullscreen my monitor's OSD shows fluctuations of about +/-15 Hz around the 72 Hz. This is just an artifact of how the monitor calculates/displays the current Hz I think. But in window mode it fluctuates from roughly 50 to 144 Hz.

Would be interesting to hear from Nvidia and AMD and Microsoft how they have implemented variable refresh in window mode.

[ghost]

It is variable, not variating refresh rate, so no.

The idea with VRR is that you can show a frame at any time the host wants. There doesn't need to be a constant refresh rate. Switching the refresh rate on demand was possible before.

[Jules-A]

I tried Freesync with my RX570 and 75hz monitor in windowed Vulkan and at first it seemed like it improved things changing fps to 72/48 (I couldn't tell any difference) but after blind testing it was basically the same (at least when comparing notoriously bad panning shots in anime). Going to OpenGL has better vysnc but has random stutters if I do anything else on my system. DX11 has a far, far superior vsync and looks like FreeSync is working for sure BUT I can't use it since an anime4k denoising shader is broken under DX11. I also can't really test fullscreen vulkan since it's still broken in MPV (https://github.com/mpv-player/mpv/issues/8008)

For now I've gone back to default fps settings since I'm not sure if 48/1.001 or 72/1.001 will cause some issues and I don't always play 23.976 content.

[lextra2]

For me, FreeSync is automatically engaged (this includes frame rate doubling via driver with content below FreeSyncs minimum rate) once I go fullscreen (hwdec=auto might be needed).

This is exactly what the OP wanted, yes?

Playback is noticeable smoother for me on my Samsung TV. Also, showing the input source info on the TV shows FreeSync as enabled (It does not while mpv is windowed, further prooving things are working as intended). This is on Windows 10 with AMD driver.

So I don't see why this thread is still open? Looks like this implementation is up to the GPU vendor, which definitely works for AMD here.

[LinAGKar]

This thread seems to be about regular frame doubling (which the driver can do), but on a related note it would be nice if mpv could automatically interpolate to the highest integer multiple of the source framerate that falls within the monitors range. So in the above example with 23.976 FPS video on a 40-120 hz monitor, mpv would insert 4 interpolated frames between each frame, bringing it up to 119.88 FPS. if you watched 25 FPS video, mpv would insert 3 frames to bring it up to 100 FPS.

Because even if you can output 23.976 directly and avoid the framerate mismatch judder, you still get the low framerate shakiness. This would let you take advantage of VRR to natively show any framerate, and still get the smoothing effect of interpolation to avoid the shakiness of low framerates. Currently, there seems to be no way of using both interpolation and VRR.

[lextra2]

@LinAGKar I see what you mean. interpolation=yes could be changed to interpolation=<value> where value is any value up to the maximum refresh rate of the display (to stay in VRR range). Sounds good to me.

And it shouldn't be hard to implement, because I would think that yes just sets the limit automatically to the maximum of the targeted display, where value would respect the manually specified value.

[LinAGKar]

@lextra2, something like that, except it should automatically interpolate to the highest integer multiple below the limit (which would vary depending on the framerate of the video), whereas the current interpolation (designed for fixed refresh rate displays) interpolates strictly to the set framerate, which may not be an integer multiple.

[PalmtopTiger]

It is necessary to indicate the minimum and maximum. For example, my monitor sync range is 48-70 Hz. 23.976 2 = 47.952 23.976 3 = 71.928 So, 2.5 is the only option for 23.976 fps.

[LinAGKar]

Of course, it would be even better if mpv supported some kind of motion interpolation, rather than mere frame blending.

[jsravn]

I also use AdaptiveSync and HDMI 2.1 VRR LG C9. LG C9 does show everything in OSD, monitor does not have such HW OSD.

@ValZapod If you're using a 120hz OLED, all you need to do is video-sync=display-resample and make sure you have any frame limiters off. That will give you 5:5 pulldown which is perfect for OLED motion clarity and should eliminate all stutter.

[jsravn]

you're using a 120hz OLED,

I also use AdaptiveSync (thus Display Port) monitor. So it is not perfect for all.

I just realized it won't be perfect either, you want it at 120/1.001 for 23.976 source. So it seems there isn't a great way other than to use the fps filter. It does feel like mpv should take care of this somehow.

[magicgoose]

Even 240fps camera likely won't really be enough to accurately judge 120hz display, because you can't ensure time alignment and so there will be some motion blur. If you prepare a synthetic video with a moving rectangle, for example, and just watch it, then a naked eye should be enough to see if it's working properly, or not. Our eyes are pretty good at judging motion smoothness. It's the accurate measurement of delay that's really hard to do by eye.

[Hrxn]

What kind of display do they support? I mean, so far it is the new Macbook Pro.. and, that's about it, I think?

[lextra2]

Yeah would be nice to get a list of supported devices for apple vrr?

[lextra2]

Apple adding VRR in such a competent way really surprised me. It is the best implementation from any plattform.