Make "sphinx" the default for 'tscale' interpolation!

[mpvOWNZ]

Example video to make my point: https://www.youtube.com/watch?v=MonFNCgK4WE [Mad Max - Fury Road "Retaliate" Trailer] Just when looking at the first half minute or so You will notice the excessive amounts of blurring that takes place with the new default of "tscale=mitchell". The same trailer with "tscale=triangle" however, is way more sharp while still being very smooth. So, for me at least, "tscale=triangle" is the perfect compromise between smoothness & sharpness (and trust me when I tell You that I've tried them all...). IMHO, mpv's defaults should satisfy the needs of the majority of users; while "tscale=mitchell" may be a little bit smoother, the blurriness it causes could scare away users that don't bother checking out each and every tscale option that there is.

I'm interested to hear what You guys think about this and whether You think that there is an even better tscale option other than triangle.

[mpvOWNZ]

I just now checked that trailer out again, just to make sure I hadn't just imagined the improvements "tscale=triangle" brings to the table... Nope, I was spot on! If You bother watching the video all to the end [it's well worth it, IMHO], both with "tscale=mitchell" (current default) and "tscale=triangle", You should also notice that one can see way more in-between, very fast cut scenes with "tscale=triangle", while also heavily cutting down on the blurness factor (which is quite a bad thing, since You also don't see that amount of blurring in the cinema, for which these kind of films are made for in the first place). In conclusion, I believe mpv should have its best options set by default, so that new users who are just checking out mpv 'as just another video player', get blown away by its capabilites, which "tscale=triangle" clearly satisfies!

Sorry for ranting quite a bit, it's just that I like mpv so much, that I want it to show off in the best possible way 'out of the box'!

[haasn]

I don't really see a big difference on this clip. Where specifically are you seeing more blurring with mitchell than triangle?

[mpvOWNZ]

@haasn First of all, thanks for replying! In the very first scene with the truck arriving; especially the foreground objects are blurred very badly, in my opinion. Also, after watching a few more very fast cut trailers, it is very clear to me that triangle allows one to see these fraction of a second scenes quite clearly, whereas with mitchell I don't even notice that they were there in the first place. Lastly, I find the smoothness with triangle right on par with mitchell. From all of this I concluded that triangle is in fact the better option for interpolation by default.

I hope more people try both mitchell & triangle out, so that we can get a better picture of what the majority of people think about this!

[Hrxn]

Will also try a comparison later...

[thiagokokada]

@mpvOWNZ If you could post a small clip exactly at the place where the blur occurs, it would help. You could use something like:

$ mpv video.mkv -o video_cut.mkv --start=0:15:00 --end=0:20:00

[mpvOWNZ]

@Hrxn Thanks for looking into it! @m45t3r I have attached a few seconds clip from the second Star Wars 7 trailer (with no audio) that has a nice panning shot. [I had to zip it for GitHub, since they don't support MKV...] With mitchell, the blur and ghosting is quite uneasing, whereas with triangle I can still see the details quite clearly (especially when concentrating on the background). I hope this clip is convincing enough... swep7_interpolation_test.mkv.zip

[sda89ha9]

i still prefer oversample all other filters are too blurry ( maybe its my monitors response time? )

[mia-0]

I think it’s impossible to pick a default that works for all users, since this is highly subjective and depends on the characteristics of the display device.

[mpvOWNZ]

@lachs0r Then why was mitchell made the new default with mpv 0.14.0 (genuine question)?

I understand that there can't be an option that satisfies the needs of all users... It's just that I think we can all agree on these facts:

• oversample: sharpest picture, but still jerky movement [doesn't really live up to the name of interpolation]
• mitchell: smoothest movement, but heavy on the blur factor [that amount of smoothness comes with a price, after all...]
• triangle: the compromise between the two; almost as sharp as oversample, while still delivering an absolutely smooth movement!

Therefore I believe that triangle is still the perfect candidate for being the default choice for interpolation. [Criticism always welcome!]

[lamarpavel]

@mpvOWNZ let's say I agree with your description of mitchell, triangle and oversample; it's still a very subjective topic. I absolutely dread the smudging artifacts introduced by both mitchell and triangle, so I prefer oversample even if it isn't as smooth. To be honest, I prefer no interpolation over mitchell because a clear picture is very important to me.

So here I am, saying that I - personally - would set oversample as default.

My point being that @lachs0r has a valid point, this is simply subjective based on the taste and perception of the viewer.

[thiagokokada]

Testing some videos myself, I am starting to like more triangle than the other option I used (gaussian, that seems to be slightly less blurry than mitchell, however more blurry than triangle). However, I can understand why this issue seems to have a low priority, since interpolation is not even enabled by default (so anyone enabling this option should, in theory, understand its consequences and select the best option by himself).

Anyway, I know that it was @haasn that implemented interpolation and maybe he has a monitor with high refresh rate (120Hz+), so maybe this is why mitchell does not look so blurry for him. I have a LG25UM65 (ultrawide monitor with 60Hz of refresh rate) and mitchell does indeed looks the blurriest of all tscale options (I tested: gaussian, mitchell, triangle, oversample). And 60Hz monitor are the most common out there.

[mpvOWNZ]

@lamarpavel That's why I sincerely asked @lachs0r why the default was changed to mitchell just recently. @m45t3r Good to know I'm not the only one! But AFAIR, even @haasn has only got a 60Hz monitor (might have changed by now).

Also, I believe that having mitchell as the default for interpolation can tarnish the reputation of mpv for users who don't bother trying every tscale option available. (Because it will look blurry for everybody, no matter how fast their refresh rate is...)

[lamarpavel]

@mpvOWNZ I agree for the most part, but as far as I can deduce from the manpage the default vo is still plain opengl and no interpolation is active. This means whoever comes in contact with the default tscale setting will have enabled interpolation manually and thus is likely able to try out different tscale settings. In that case we are already talking about a rather technical audience.

Please correct me if my assumptions are wrong.

[CamilleScholtz]

I've been using triangle tscale for a long time. People on imageboards have told me that it's a bad tscaler for some reason, but to me it looks like the absolute best one. oversample is too jittery, mitchell is too blurry.

[thiagokokada]

Just FYI, did some more tests. I watch anime mostly, so I decided to test using anime sources. And I basically tested all tscale filters available.

The best result I got was actually catmull_rom. It is similar to triangle, however I had less visual artifacts. triangle obvious gets second place. gaussian works somewhat well too, more blurry than catmull_rom and triangle, however not as much as the rest. mitchell is really blurry, just like ginseng and robidoux family. oversample is not very smooth at all, however it is really sharp. spline family, box and nearest are unusable thanks to the strange artifacts.

[mpvOWNZ]

@m45t3r First of all, thanks for sharing Your findings with us! My impression of catmull_rom is that it leads to a slightly sharper picture, while also being less smooth and thus more jittery when compared to triangle. Is this something that You can confirm?

@onodera-punpun Thanks for Your support! However, could You too (everybody else is welcome either) test out catmull_rom and tell us what Your impressions are? I imagine with You being a long time user of triangle, that You could more easily spot any differences. Thanks in advance!

[Hrxn]

Okay, I made some test runs with the short clip linked earlier as well

swep7_interpolation_test.mkv.zip

Since no one seemed to bother, what are you using as vo? --vo=opengl:interpolation=yes:tscale=XXXX or --vo=opengl-hq:interpolation=yes::tscale=XXXX

Or is using opengl-hq just assumed, as an unwritten rule, so to say?

Also, if you use any other video options or filter, you maybe should state that as well.

BTW, is there any difference between using --vo=opengl:interpolation=yes and --vo=opengl:interpolation ?

My result so far:

When it comes to smoothness, I can't barely notice a difference, to be honest. Maybe my display is just crap. None of it is what I would call really super smooth. No drops and stuttering, the frame rate is just not high enough to be really smooth, in my opinion.

Additionally, I think that short scene is a bit too fast, maybe a slow pan across some horizon or something like that would work better.

Otherwise, speaking of sharpness and overall impression of image quality, I would rank them like this, in descending order: oversample, triangle, catmull_rom, mitchell (Did not try the others yet)

That's basically the same as you were seeing, more or less.

[mpvOWNZ]

@Hrxn Thanks for also looking into this! First, let me try to answer Your questions:

• I indeed do use opengl-hq, however with these additions / modifications:
• "haasnsoft" for both scale & cscale, and "fbo-format=rgb32f" + "tscale-clamp"
• No, there shouldn't be any difference whether You set 'interpolation=yes' or just 'interpolation'
• BTW, You're also setting "video-sync=display-resample", right? (Else interpolation doesn't work at all)

As another test case, I also added a half-minute clip from the opening scene of Harry Potter 5 [no, I'm not a fan!] which has a slow panning scene with moving cars. To be honest, I still can't really decide between catmull_rom & triangle, but I still find catmull_rom a little bit jerkier than triangle...

Hopefully more people can chime in, as it seems from the comments so far that the only thing left to decide is whether catmull_rom or triangle is the better option, but definetly not mitchell (or going back to oversample)! hp5_interpolation_test.mkv.zip

[Hrxn]

BTW, You're also setting "video-sync=display-resample", right?

Yes, I did. I tried some different settings for video-sync as well, but they look all the same. If I understand it right, this should not affect the video output anyway, so I guess that is not a surprise.

I indeed do use opengl-hq, however with these additions / modifications: "haasnsoft" for both scale & cscale, and "fbo-format=rgb32f" + "tscale-clamp" [...]

Thanks, good to know. By the way, I think just pasting the string would be easier ;-)

[mpvOWNZ]

I found two more videos which make a good test case (and can directly be opened, too): http://www.spirton.com/uploads/InterFrame/20110618-Sample-Original.mkv http://www.spirton.com/uploads/InterFrame/20130218-Sample-Original.mkv

For reference, here are the same videos but converted to true 60fps (with InterFrame): http://www.spirton.com/uploads/InterFrame/20110618-Sample-InterFrame.mkv http://www.spirton.com/uploads/InterFrame/20130218-Sample-InterFrame.mkv

When watching both original samples with catmull_rom, I can't help but feel a certain uneasiness... It's hard to explain, but I believe it has something to do with what @haasn explained about interpolation and recreating some 'intermediary' pixels:

• With oversample, one sees both sharp (i.e. unaltered) and blended frames, which look like irregularites for our brain not unlike the 3:2 pulldown stutter one notices with no active interpolation.
• tscale options like catmull_rom, triangle & mitchell however blend each and every frame into the next one, so we basically see only blended frames, which can help to trick our brain into believing that the movement is smoother than it actually was, since the pixels are overlapping between frames and thus creating new pixels which weren't there in the original video. (And which are also the source for all the blur.)
• Now, the smoother the picture these filters produce, the blurrier the result consequently has to be.
• Therefore, while mitchell may be the smoothest of these three, it's also the blurriest.
• catmull_rom, on the other hand, while being the sharpest, is also jerkier than the other two, which makes sense.
• Finally, triangle still being the best of both worlds and thus the most adequate compromise.

If there's anything wrong with my understanding of the whole matter, please don't hesitate and do point it out, as I'm eager to learn more about this intriguing matter of interpolation and watching videos smoother than I even could in the cinema!

[habaneros]

Which one of these filters is easier to compute, catmull_rom or triangle?

BTW, what do you mean, 'smoother than in the cinema'?

[mpvOWNZ]

@habaneros That is a good question. I remember @haasn stating that triangle is much easier to compute than mitchell, but I don't know how triangle compares to catmull_rom... BTW, which one do You find better? Or rather, which filter do You use for tscale?

Also, what I meant with 'smoother than in the cinema' (I admit I should have made this clearer from the beginning) is that when I watch a video with 23.976 fps with mpv with activated interpolation (with tscale=triangle), then it feels smoother and easier on the eyes than the same film in the cinema. Maybe it's just me, but ever since I discovered mpv and all the benefits it brings, I have been kinda spoiled and find myself wishing that I could just hook up a laptop to the projector in the cinema and show everyone what they are missing out! ;-)

In the end, I think it just has to do with the fact that 24fps really is not ideal for showing smooth movement, however what I dread even more is the soap-opera effect that shows itself with movies shot in HFR (like the Hobbit trilogy). Therefore, I believe mpv with interpolation provides the best viewing experience, by combining the traditional film effect with smooth movements.

Hope that clears it up!

[CamilleScholtz]

It might be placebo, but I think I like catmull_rom a bit smoother but still sharper than triange. Both are a lot better than oversample/mitchell.

[ghost]

Which one of these filters is easier to compute, catmull_rom or triangle?

mitchell and catmull_rom have the same speed, triangle is faster. oversample might be the fastest.

[Hrxn]

Both are a lot better than oversample/mitchell.

This makes me question your definition of 'a lot', to be honest ;-)

mitchell still works fine, in my opinion, as well as oversample. It's not like lanczos or nearest, they gave me noticeable block artifacts on some files.

The thing is, as long as we have source material with 24 fps, it will never be really smooth. Time to switch to 60 fps everywhere ;-)

BTW, what does it mean when {avsync} and {total-avsync-change} both return "unavailable"? Shouldn't that be a number, zero for example, if it's working?

[ghost]

Probably playing without audio?

[Hrxn]

Yeah, spot on..

That mkv file only has one video stream..

EDIT:

And I did not notice because my PC was muted the whole day ;-)

[kevmitch]

oversample has the distinct advantage of leaving native frame rate content unblended

[ghost]

Well, we could add an option that disables interpolation for same-framerate content.

[mpvOWNZ]

@wm4 Thanks a lot for clearing the situation around the different filters up! This probably also explains why I was sometimes seeing framedrops with catmull_rom (same with mitchell) while triangle showed no problems. I guess a NVIDIA 750 Ti with opengl-hq output and "fbo-format=rgb32f" + "scale=haasnsoft" can't handle the load (even with activated hardware decoding)...

@onodera-punpun Thanks for sharing Your impression of catmull_rom! Care to share which GPU You are using (and whether You use 'hwdec')? Still, if a filter can produce about the same results but with less computations, then that's still a win in my books! [Yes, I'm well aware of my bias here ;-}]

@Hrxn I think Your 120Hz monitor is making all the difference compared to the more widespread 60Hz. Also, am I right with my hunch that You are used to watching 24fps content interpolated to true 60fps? That might explain why You find mpv's interpolation not really smooth...

[Hrxn]

Yes, I agree that an 120Hz monitor is not ideal for any sort of moving pictures at less than 60 fps.

I compared the test files on another monitor and on my laptop, and the impression of the image is better there, less flaky and jittery. Might be subjective, though. Or the vsync jitter in my example is too high, I don't know what's a good value here...

The higher PPI/DPI of a laptop screen also helps a lot, in my opinion.

I am not really that bothered by 24 fps content, but I, for example, always notice the stuttering image while panning during a long shot when I'm in the cinema.

Compared to that traditional film stuff, I really like the smoothness of 60 fps, especially with motion in the image.

To be clear, I see the difference between interpolation on and interpolation off, but it does not come near the smoothness of real 60 fps, in my opinion.

That's what I tried to express with 'not really smooth'..

Well, we could add an option that disables interpolation for same-framerate content.

I completely support this notion!

[haasn]

mitchell and catmull_rom have the same speed, triangle is faster. oversample might be the fastest.

oversample is the fastest for sure. triangle could be faster if people like it, the current implementation goes through the same (slow) generic mechanism as the other filters, but we could make it much faster. (similar to bicubic_fast vs bicubic)

[ghost]

By the way, I made interpolation disable itself if video=display rate, see tscale-interpolates-only sub-option. I hope this doesn't cause any problems, because sometimes display-sync will only schedule 1 vsync for a frame to compensate for timing errors etc.

[bodayw]

In madVR if you enable "smooth motion" the default sub-option says "only if there would be motion judder without it...", and madshi explains it as "...Basically it's tuned so that FRC is only enabled if e.g. the framerate is 24.000 and the refresh rate is 24.500. Any less difference than that and FRC is turned off."

Maybe we should make ``tscale-interpolates-only" default as well?

[mpvOWNZ]

@haasn I for one would very much appreciate it if triangle could be made even faster, since by the looks of things, I'm going to be using it for the foreseeable future. Just to be sure: this speedup to the algorithm wouldn't cause any quality degradation, right?

Thank You very much in advance!

[ghost]

Maybe we should make ``tscale-interpolates-only" default as well?

It was default the moment it was added.

In madVR if you enable "smooth motion" the default sub-option says "only if there would be motion judder without it...", and madshi explains it as "...Basically it's tuned so that FRC is only enabled if e.g. the framerate is 24.000 and the refresh rate is 24.500. Any less difference than that and FRC is turned off."

I don't know if this is the same as with the new mpv sub-option. Maybe not. In mpv, interpolation will be disabled simply if a video frame is scheduled for exactly 1 vsync display. If you play 23.976 fps video on a 24 Hz display, and you use video-sync=display-resample, then interpolation will be disabled, because the video is speed up to 24 fps to match the display refresh rate. (I'm not sure if madVR's description implies the same, or if it actually has some sort of 0.5 threshold.)

[ghost]

And obviously my approach was less than ideal anyway, so I've replaced it with some sort of threshold too: 34bead485987b416685e73ca918610fff75d618d.

[mpvOWNZ]

Am I even right with my assumption: 'the blurrier the images are, the smoother the resulting movement; the sharper the images are, the jerkier the movement'? If not, could there be a sort of "one tscale filter to rule them all" [and in the darkness, bind them ;-]? I mean, a filter that produces extremely smooth movement while at the same time retaining the original sharpness of the picture without producing any noticable blur or other artifacts? Or is this something that is fundamentally opposite to each other?

[lamarpavel]

@mpvOWNZ, not fundamentally opposed, but related. However, you need to include computation cost as related parameter if you want to create some sort of wighted comparison.

[thiagokokada]

@mpvOWNZ there is the approach used by SVP (Smoothmotion Project), known as motion-based interpolation that results in a very sharp image, while being really smooth. This approach actually recreates the "missing" frames, however there is a different number of issues and obvious artefacts for using this technique.

[0xBRM]

@m45t3r It looks good if you increase the interpolation threshold so that the confidence coefficient of motion vectors is high. It doesn't interpolate as many frames, but it results in considerably less artefacts, and better panning shots.

[thiagokokada]

@CrisBRM I am not saying that the result is bad, it looks quite good (better the mpv interpolation IMO), however there is artefact independent of the settings.

One problem I was having with SVP, for example, was with motion scenes including text (Anime OP/ED, or just hardcoded subtitles), that would make the text duplicate and was quite annoying and distracting. Since I was basically using two different players (mpc-hc with SVP and mpv) because of the issue above, I simple uninstalled SVP and got back to mpv.

I did some more tests btw. I can confirm that using catmull_rom with less capable hardware results in jerky motion. Actually, to get a good result using catmull_rom, I need to use backend=angle (or maybe backend=dxinterop, however NVIDIA Optimus still does not work with backend=dxinterop as reported in issue #2620), since using backend=win and catmull_rom results in a worse image than triangle. With backend=angle, however, I do think that catmull_rom offers the best image of all scale filters, sharp and smooth. I need to test the same findings on Linux too, however I generally don't watch too much videos in my Linux system (don't have a monitor as nice as my Windows system).

IMO, I think triangle would be a nice default considering that it gets a good result and low hardware utilization and works great even with backend=win. I still think catmull_rom, properly configured, works better, though.

[thiagokokada]

However, now I understand why @haasn didn't see much difference between filters. It seems when you have a proper backend (backend=angle or backend=dxinterop, since OpenGL drivers, as far I understand, are quite broken in Windows), the difference between then are not significative. Even mitchell looks great with backend=angle. Using the default backend (backend=win) the difference are much more expressive, because the artefacts are much more visible. Since @haasn uses mainly Linux, I don't think he experience this kind of problems.

Maybe the default backend should change, however as I said backend=dxinterop is broken in some setups, while angle is not as featured as the other backends.

[ghost]

I can confirm that using catmull_rom with less capable hardware results in jerky motion

If it only happens with "less capable" hardware, then the driver might delay frames and skip vsyncs. The delayed frame counter is supposed to detect this.

Using the default backend (backend=win) the difference are much more expressive, because the artefacts are much more visible.

That shouldn't happen.

[mpvOWNZ]

@m45t3r I found Your findings really interesting and intriguing, to say the least. Even though @wm4 says the backend shouldn't have any impact on the quality (which kinda makes sense), You apparently experienced otherwise, which means there must be some kind of variation, at least when it comes to the frame-blending operations with interpolation. After reading up on what ANGLE is, here is my hypothesis for the possible difference:

• Direct3D must do something differently than OpenGL, which would mean that only Windows users are seeing this. (It would be great if You or anyone else who also has access to Linux could check; unfortunately I'm not using Windows, else I would have tried this out myself already...)
• ANGLE utilises EGL, whereas my NVIDIA GPU on Linux uses GLX/X11. If someone who uses MESA on Linux could test with both EGL & GLX to see if that makes any difference with tscale filters, that would be really great!

It's especially interesting that You found tscale=triangle better with backend=win ['pure' OpenGL with WGL], but on the other hand catmull_rom with ANGLE better [either because of Direct3D or EGL]. That would also explain why I still think that triangle looks better for me than catmull_rom [neither Direct3D nor EGL]. For any feedback I would be really grateful!

[kevmitch]

The performance of different back-ends can be significant. One measure is this is the vsync-jitter property. A large value for this indicates that accurate time interpolation may be difficult.

Playing a 23.976fps clip on a 59.94hz display, I get the following results

x11	x11egl	win	angle	dxinterop
0.045	0.05	0.5	0.007	0.004

unfortunately, i don't have a dual boot machine, so the Windows results are from Intel Haswell graphics and the Linux results are from Ivy Bridge graphics. With that caveat in mind, I find

dxinterop ~> angle > x11 ~> x11egl > win

in otherwords, backend=win is terrible and producing consistent vsync timings.

[ghost]

Also mind the vo-delayed-frame-count property. If it increases during playback all is lost, and either your opengl settings are too heavy, the display refresh rate was misdetected, or the vsync timings are hopeless. Occasional delays might be tolerable, and interruptions like switching fullscreen etc. will also cause them.

consistent

inconsistent?

[ghost]

PS: angle EGL and Linux EGL are completely different things. They just superficially have the same API. (EGL is what I call pseudo-portable... we have half a dozen of EGL context creations in our code, but almost none of it can be shared.)

[kevmitch]

inconsistent?

sorry

in otherwords, backend=win is terrible at producing consistent vsync timings.

[mpvOWNZ]

@kevmitch Thank You very much for doing this comparison between these different backends! What I find astonishing is that both ANGLE & dxinterop are many times faster than X11 & x11egl (and that EGL doesn't make any difference at all). So this means that only Windows users can enjoy these performance benefits for now - hopefully Wayland can change the situation for us Linux users... Or maybe it's the fault of OpenGL after all, so that would mean that only Vulkan could enhance the performance on Linux.