Make "sphinx" the default for 'tscale' interpolation!

[mpvOWNZ]

Example video to make my point: https://www.youtube.com/watch?v=MonFNCgK4WE [Mad Max - Fury Road "Retaliate" Trailer] Just when looking at the first half minute or so You will notice the excessive amounts of blurring that takes place with the new default of "tscale=mitchell". The same trailer with "tscale=triangle" however, is way more sharp while still being very smooth. So, for me at least, "tscale=triangle" is the perfect compromise between smoothness & sharpness (and trust me when I tell You that I've tried them all...). IMHO, mpv's defaults should satisfy the needs of the majority of users; while "tscale=mitchell" may be a little bit smoother, the blurriness it causes could scare away users that don't bother checking out each and every tscale option that there is.

I'm interested to hear what You guys think about this and whether You think that there is an even better tscale option other than triangle.

[ghost]

It's not slower on Linux, it just has shitty vsync timings. For what's it worth, I get 0.005 on Linux with x11/nvidia.

[mpvOWNZ]

@wm4 How can I check the vsync-jitter timings out myself? Thanks!

[kevmitch]

@mpvOWNZ larger vsync jitter I see on Intel Linux isn't the end of the world, display sync and interpolation still works pretty well. I don't think I can visually tell the difference. My main point was that the win backend has issues, one of which is a vsync jitter at least 10 times the other backends (at least on intel).

You can use an argument like --osd-msg1='${vsync-jitter}' to display the vsync jitter during playback.

[sda89ha9]

windows 10 / amd radeon r9 290 vsync-jitter is much lower with glfinish enabled visually i can't tell the difference

--fullscreen --video-sync=display-resample

--vo=opengl-hq:interpolation:backend=win:dwmflush=no
vsync-jitter: 1.191 | mistimed-frame-count: 1 | vo-delayed-frame-count: 0 | vo-drop-frame-count: 0

--vo=opengl-hq:interpolation:backend=win:dwmflush=no:glfinish
vsync-jitter: 0.004 | mistimed-frame-count: 2 | vo-delayed-frame-count: 1 | vo-drop-frame-count: 0

--vo=opengl-hq:interpolation:backend=win:dwmflush=yes
vsync-jitter: 0.003 | mistimed-frame-count: 1 | vo-delayed-frame-count: 1 | vo-drop-frame-count: 0

--vo=opengl-hq:interpolation:backend=win:dwmflush=yes:glfinish
vsync-jitter: 0.004 | mistimed-frame-count: 2 | vo-delayed-frame-count: 1 | vo-drop-frame-count: 0

[mpvOWNZ]

I was in the process of preparing a rather big comparison of different tscale filters with regard to their effect on vsync-jitter... (Thanks @kevmitch for the how-to!) But then I noticed I wasn't running the newest NVIDIA drivers, so I updated to 361.18... Suddenly, all my performance problems and that I sometimes had to start a video file multiple times in order for it to play correctly completely disappeared! Then, after some more testing, I realised that @m45t3r was right:

• catmull_rom is the best tscale filter! (At least until @haasn or @wm4 come up with something even better ;-])

Therefore, I would like to explicitly thank @m45t3r for pointing it out in the first place! (Anyone else who has taken part in this conversation too, of course.) Also sorry to anyone who I may have bothered or troubled for no apparent reason, but I do hope this discussion has helped at least in some way...

I urge anyone who hasn't tried tscale=catmull_rom yet to do so, since it really is the best of both worlds: smooth & sharp! Plus if @lachs0r could think about making it the default for interpolation, that would be great!

[Hrxn]

@sda89ha9 Why didn't you include backend=dxinterop in your comparison?

And while we're at it, how does..

direct3d_shaders  : Direct3D 9 Renderer (using shaders for YUV conversion)
direct3d          : Direct3D 9 Renderer

compare against opengl with backend=dxinterop?

[sda89ha9]

windows 10 / amd r9 290

	vsync-jitter	mistimed-frame-count	vo-delayed-frame-count
opengl-win	1.054	24	27
opengl-hq-win	0.691	2	1
opengl-hq-glfinish-win	0.003	0	0
opengl-dxinterop	0.002	0	1
opengl-hq-dxinterop	0.002	0	1
opengl-angle	0.004	0	0
opengl-hq-angle	0.005	0	0
direct3d	0.004	0	0
direct3d_shaders	0.003	0	0

[ghost]

1.054 is excessively bad, 0.691 is still too bad. (What is going on there... probably real skipped vsyncs.)

[thiagokokada]

@Hrxn AFAIK, direct3d_* does not support interpolation, so it does not matter for this issue. The only real vo for quality is OpenGL, the others are mainly there for compatibility and special purposes.

I think this is one of the motives that ANGLE is implemented. direct3d_* may be deprecated in the future in favor of ANGLE.

[Hrxn]

The only real vo for quality is OpenGL, the others are mainly there for compatibility and special purposes.

I was not really aware of this, I thought Windows would be the exception here. Thanks for the heads-up!

[mpvOWNZ]

@lamarpavel @sda89ha9 Have You already tried out tscale=catmull_rom and if so could You report how it looked? Since both of You said that You're still prefering oversample because of the sharpness, I'm wondering if You are still noticing any blurring or other artifacts with catmull_rom? For me, there is practically no visible difference between both filters, with the added bonus of better smoothness that comes with catmull_rom, so I personally see no reason to keep on using oversample.

@Hrxn I remember that You were preferring both oversample & triangle over catmull_rom on Your 120Hz monitor. Is that because You were seeing jitter or any other artifacts? I'm asking because I'm wondering whether catmull_rom works equally well on a 120Hz monitor as it does on a 60Hz one. (You know, for the future and stuff :-])

[Hrxn]

No, no noticeable jitter, at least not something I could see with my eyes. I will try some measurements.. And definitely no artifacts. With some scenes I liked triangle better, others catmull_rom. It's too close to call, really. Both worked fine, but subjectively, I'd say catmull_rom is a bit better. oversample is another story, you get more sharpness and crispiness but lose some smoothing effect and that blurring of motion look. Personally, I liked oversample in all tests, but I really think this one comes down to individual preferences of trade-offs.

[lamarpavel]

@mpvOWNZ Comparing oversample and catmull_rom directly the latter feels noticably more blurred to me, but that can very well be a placebo.

A good sample to test is Django unchained between 1:10 and 1:30, where there is a lot of horizontal movement. Making screenshots reveals that the type of blurring is vastly different (as you would expect), but still images are somewhat worthless here as we are talking about an effect that is meant to improve the perception of moving images. My conclusion is that I can not objectively say which one is better but subjectively I think I notice more blurring with catmull_rom. However, I don't notice much difference in smoothness.

I'm testing on two setups here:

1. Archlinux, X11, mesa, amdgpu (yes, the new driver), vdpau, 60 Hz
2. Archlinux, X11, mesa, intel (sandy bridge), vaapi, 60 Hz

Is it possible to switch the tscale filter online? In that case we could write a Lua script that helps us do ABX tests.

[kevmitch]

Is it possible to switch the tscale filter online? In that case we could write a Lua script that helps us do ABX tests.

you can use the command vo-cmdline.

 mp.commandv('vo-cmdline', 'interpolation:tscale=oversample')

[lamarpavel]

With the script

local interp_changed = true

function toggle_interp()
    if interp_changed then
        mp.commandv('vo-cmdline', 'interpolation=yes:tscale=oversample')
    else
        mp.commandv('vo-cmdline', 'interpolation=yes:tscale=catmull_rom')
    end
    interp_changed = not interp_changed
end

mp.add_key_binding("T", "toggle_interp", toggle_interp)

I tested various sources and for the most part can't tell which is which. I do recognize a difference, but I can't tell which filter is active with certainty. I will do more tests with different source material before I jump to conclusions, in the mean time others might try the script and do some blind tests.

Note that the script works, as I can clearly distinguish oversample and ginseng and identify them 100% positively, which is as clear a result as an ABX test can deliver.

[mpvOWNZ]

@lamarpavel Wow, really awesome work with the script and definitely the best way to come as close as possible to an objective conclusion about this whole matter! I will try this over the coming days and report my findings once I feel confident enough. Thanks alot for coming up with this! (And @kevmitch for pointing out the direction to take.)

@Hrxn Hopefully I'm not asking too much, but it would be really great if You could do the same ABX test with Your 120Hz monitor, since I'm right now in the process of deciding whether it would be worth to get one, too. Thanks in advance!

@m45t3r Since You were saying that with backend=ANGLE on Windows even mitchell looked great, does that mean You are not seeing blurring with it any more? Because on Linux with backend=x11, I still certainly do...

[OttoKurschner]

It would be interesting to test the different filters without the H.264 loop filter (one nice feature of the old SMPlayer that I never saw anywhere else)

[ghost]

without the H.264 loop filter (one nice feature of the old SMPlayer that I never saw anywhere else)

You can disable that with ffmpeg options (--vd-lavc-o).

[OttoKurschner]

Thanks wm4, it appears you get slightly better motion when skipping Bidirectional interpolation but there is a noticeable tradeoff in "texture quality" on faces and walls, I would never recommend to try this without a minimum of debanding+interpolation+display resample (actually probably not at all) but it is interesting how the decoder might see some improvements in this area

vd-lavc-skiploopfilter=bidir

Back on topic, I have gone back to hermite radius 3, at least to my to my eye it is unmistakably superior to catmull rom and mitchell on my particular displays (all phosphor type) but I can also understand the appeal of a sharper cubic if you have a slow-ish GtG monitor coupled with RTC/Overdrive or something

I am interested in what others think about hermite

[haasn]

I think the placebo of having changed the value is more than any actual differences. (And I'm certainly guilty of this myself)

[mpvOWNZ]

I just noticed that haasn has switched to tscale=catmull_rom, too! So maybe this could be the final tipping point... https://github.com/haasn/gentoo-conf/blob/nanodesu/home/nand/.mpv/mpv.conf

@haasn `#brightness=2 # Compensate for black clipping

contrast=-2 # Compensate for white clipping`

I think a better way to do this is to change between Full & Limited RGB in the GPU driver (as said in the mpv manual).

[haasn]

I just noticed that haasn has switched to tscale=catmull_rom, too! So maybe this could be the final tipping point...

I'm just rotating the value of tscale regularly to make sure I get the best of all worlds.(tm)

[mpvOWNZ]

@haasn Ah, I see! :) Also, @OttoKurschner said that he uses tscale=hermite, but I don't see this in the list of available scalers; does that mean one can use pretty much any scaler, even if not listed with mpv --vo=opengl:tscale=help?

Plus two more things I noticed:

• You are setting your very own haasnsoft to scale-radius=3, whereas in mpv it is around 3.24. If the value You are setting is indeed better, why not change the default value too, so we can all benefit from it by default?
• You are using temporal-dither; doesn't this lead to noticeable flicker? And what are the benefits?

I hope I'm not bothering You, but it would be cool if You could answer these questions, as I'm sure others would be interested in the answers, too! (They just don't know it yet... [tm])

[haasn]

Also, @OttoKurschner said that he uses tscale=hermite, but I don't see this in the list of available scalers; does that mean one can use pretty much any scaler, even if not listed with mpv --vo=opengl:tscale=help?

No, you can't. But you can design your own B/C-spline using e.g. tscale=bcspline and picking the values with tscale-param1/2. You can also use some more filters via tscale=box:tscale-window=help.

You are setting your very own haasnsoft to scale-radius=3, whereas in mpv it is around 3.24. If the value You are setting is indeed better, why not change the default value too, so we can all benefit from it by default?

It's not better, but it is faster. The higher the radius, the more exact the transformation - but the slower it gets. My GPU just can't handle radius 3.22 on 4K 60 Hz input clips, so I turned it down since the visual difference is not very big.

You are using temporal-dither; doesn't this lead to noticeable flicker? And what are the benefits?

I can't see any flickering at dither-depth=4, let alone dither-depth=8. I can only see flickering at very low dither depth, and that might just be due to the depth rather than some inherent property of the temporal dither mask.

Temporal dithering eliminates static dither masks in favor of more random looking noise. I prefer that, honestly.

[Hrxn]

Yes, I am interested in some answers as well, thank you for helping out ;-)

By the way, is there another reference manual I don't know of yet? I've only used this one, linked on mpv's site: https://mpv.io/manual/master/

Useful, but not really complete, it seems, because hermite or minimum-scale aren't even mentioned there, for example.

Oh, and I noticed that you set scaler-resizes-only, which should already be the default, according to the manual, is there any reason for this, or is it only some relic from the past?

[mpvOWNZ]

@haasn Thanks a lot for taking the time to answer my questions and helping us all out!

@Hrxn That's the only resource I'm aware of as well (plus GitHub of course). And yes, scaler-resizes-only is the default since mpv 0.16.0. But haasn used to (maybe still?) maintain a fork of his own called "mpv-hq", which could be the reason for still setting the option.

[mpvOWNZ]

@haasn tscale=bcspline:tscale-param1=0:tscale-param2=0.6283185:tscale-radius=2 Sorry if it appears as though I'm stalking You (I don't, I swear!), but I see that the above is what You use now for interpolation. As I'm sure that You put a lot of effort into finding this sweet spot, I thought that it would be beneficial for all of us mere users of mpv if You could compress all of the above settings into a new scaler for tscale, similar to what You have already done with haasnsoft, so that we all could try this out more easily and maybe even make this new scaler (haasnpolation?) the new default for mpv. Thanks in advance!

[haasn]

Uh no, I just copied those values from somebody (for testing) and never bothered to change it since then because I pretty much just forgot about it.

I'm fine with whatever default. I have yet to see a difference.

[bodayw]

I'm fine with whatever default. I have yet to see a difference.

Then I guess better just set the least computational demanding one as default, if that make any sense?

[haasn]

@bodayw They're all equivalent (at least for the ones based on convolution)

That is mitchell, catmull_rom, bcspline etc. oversample is faster but also definitely noticeably worse (IMO).

[haasn]

I looked at the test clip linked earlier again and I see no difference between mitchell, catmull_rom, and the settings in my mpv.conf at all.

bicubic is slightly blurrier, and triangle is excessively, ridiculously blurry. oversample is less blurry but more stuttery, and nearest is the least blurry but the most stuttery (obviously).

Based on this I say go for mitchell or catmull_rom. If it helps people whose monitors have lower response times, I wouldn't have anything against changing the default to catmull_rom. On my IPS panel I see no difference.

Edit: Oops, I was accidentally testing with a custom value of tscale-radius=2, which is why triangle looked so wrong/weird for me. I think now I can't even see a difference between triangle and mitchell. Based on this, a case could be made for setting triangle as the default, since it's more efficient than the others. (And, in fact, a specialized version could be made which is very efficient)

[Argon-]

As user with an iGPU who considers triangle pretty good and oversample way worse, I'd appreciate a very efficient triangle.

[haasn]

@Argon-: Well, have a very efficient triangle.

[mpvOWNZ]

Even though I initially pledged for tscale=triangle to be the new default, I was later on proven wrong by @m45t3r insofar that he found out that tscale=catmull_rom really is the best option of all the available scalers. It produces the least amount of blur while still providing very smooth panning shots. (Good example would be the ones seen in "The Revenant".) Maybe the difference is not very noticeable on panels with slower response times, but it would be good to hear whether people with faster panels (like the newer G-Sync / FreeSync types) are able to tell the difference between the different scalers more easily.

[CamilleScholtz]

does linear look exactly the same as oversample, just using less resources? That's what it looks like from the code to me..

[haasn]

linear looks exactly the same as triangle, not oversample.

[haasn]

I found a good example of a situation in which mitchell performs significantly better than both linear and catmull_rom: The slow-scrolling white-on-black credits at the end of a movie (in particular, Citizenfour).

With linear, the white text appears to flicker quite strongly, which is an expected result of the way linear mixing adds aliasing. With catmull_rom, it still aliases but to a lesser degree. With mitchell, the text is almost perfectly smooth, with only very slightly noticeable flicker.

Edit: With bicubic or gaussian, the text is perfectly smooth, but slightly blurrier.

I'm confident I could ABX between bicubic/gaussian mitchell, catmull_rom and linear on this sample. (But I don't think I could ABX between bicubic and gaussian)

Finally, with the bcspline configuration above, I get flickering similar to the type observed with catmull_rom.

Edit 2: After swapping back and forth between bicubic and gaussian during playback for a while, I observed that bicubic is ever so slightly blurrier but smoother than gaussian. I reckon I could ABX between them after training, on this particular clip.

To summarize my findings, I notice a clear trend between sharp-but-flickering and smooth-but-blurry, which goes something like this:

oversample <-> linear <-> catmull_rom <-> mitchell <-> gaussian <-> bicubic

So, pick your poison I guess? Either way, mitchell is pretty near the center of the spectrum, and based on picking a default I think it should be either catmull_rom or mitchell on these grounds, not something extreme like linear.

[Argon-]

@Argon-: Well, have a very efficient triangle.

Nice, thanks!

[lamarpavel]

@haasn I agree with either mitchell or catmull_rom as default. We should identify which of both causes more obvious artifacts in common source material (grainy film, digital film, scanned animation, cgi-animation, anime, game/screen recordings) and then chose the other. Hopefully resulting in a good middle ground with less users noticing artifacts.

Also, would it make sense to have a default switch based on screen refresh rate? It seems to be a common source for difference in filter performance (quality wise) and it should be more easily detectable than the type of image (animated vs film etc).

[haasn]

Given the amount of (most likely placebo) support for catmull_rom, might as well just set it as the default. I think it's very similar to mitchell (both in appearance and in the actual curve itself, if you graph it out), but apparently people disagree.

I don't think there's a good reason to have performance-dependent automatic switches in mpv. If we did, it would have to affect much more than just tscale (this would be equivalent toe e.g. choosing the default scaler based on screen resolution and GPU performance)

[mpvOWNZ]

@haasn How come tscale-clamp can lead to even more blur? And isn't it desirable to always avoid those ringing artifacts / black flickering around moving objects? If so, shouldn't tscale-clamp be activated by default?

[CamilleScholtz]

and isn't it desirable to always avoid those ringing artifacts / black flickering around moving objects? If so, shouldn't tscale-clamp be activated by default?

I think it is because those ringing artifacts / black flickering don't happen with every monitor, I've never seen any for example, and don't use tscale-clamp.

[mpvOWNZ]

@haasn The reason why I'm asking whether tscale-clamp should be activated by default is because the current default of tscale=mitchell is also showing this black flickering. But how come eliminating that leads to even more blur? And what about that pixel overdrive that You had mentioned before?

[mpvOWNZ]

After a lot of careful consideration, I decided to close this issue and leave the default of tscale=mitchell be.

Here's why:

Flickering is more annoying than blurriness, and also something one doesn't experience in real life, whereas motion blur is a very well known phenomenon. tscale=catmull_rom, however slight, does produce still noticeable flickering, especially during close-up shots, when the actors heads just move a little, which can look like encoding errors to the uninitiated. Thus, I decided that the current default really is the best option for the most amount of users. Nevertheless, I hope this whole discussion was futile enough and we all learned a thing or two! So, until someone comes up with a tscale option that sits right between catmull_rom & mitchell, no further arguing is required.

PS: I still believe tscale-clamp should be enabled by default when setting interpolation... ;-)

[CamilleScholtz]

After a lot of careful consideration, I decided to close this issue and leave the default of tscale=mitchell be.

IMO you should really leave this issue open, there are many people who've joined this discussion now, and isn't really "your issue" anymore because many people seem to think linear or catmull_rom is better :).

Flickering is more annoying than blurriness, and also something one doesn't experience in real life, whereas motion blur is a very well known phenomenon.

I disagree, I'm even using oversample because I can't stand the blurryness of mitchell.

[mpvOWNZ]

@onodera-punpun İf You are usıng oversample, then You are not goıng to see flıckerıng at all, because oversample alternates between sharp and blended ımages, whereas all the other scalers actually blend every frame ınto the next. Therefore, You can't even ımagıne how annoyıng even slıght flıckerıng can be... :8ball:

[mpvOWNZ]

Seems this option has only recently been added to mpv, as I can't remember trying it out before... Anyway, I think the results look pretty good (minimal artifacts & motion blur while being pretty smooth). I don't know how computationally intensive this option is, but if tscale=sinc is less taxing than the current default of tscale=mitchell, then I think it deserves to be promoted as the new default behavior! (Plus the added benefit of producing noticably less motion blur than tscale=mitchell.)

@haasn Could You please test this on the end credits of "Citizen Four" and share with us Your findings?

Thanks in advance!

[thiagokokada]

I think we ae lacking a good set of samples (maybe 10 or 20 small videos up to 10 seconds each) to test this filters. It would be interesting to have these kind of samples stored somewhere with different kind of videos (movies, animes, CGI, etc.) to make it easier to decide which filters works best in the majority of cases.

[sda89ha9]

@mpvOWNZ looks like tscale-clamp is enabled by default see https://github.com/mpv-player/mpv/commit/6ff1cd5502e4f88783c4bb615164b875511d9b71 and https://github.com/mpv-player/mpv/blob/master/video/out/opengl/video.c#L314

[sda89ha9]

so it seems ffmpeg has new motion interpolation filter https://ffmpeg.org/ffmpeg-filters.html#minterpolate too slow for any realtime playback