mpv shaders : blurring 、wrong color conversion and freezing

[hooke007]

state:

• [x] blurring
• [x] color conversion
• [ ] freezing (d3d11) 👉 workaround https://github.com/AN3223/dotfiles/issues/6#issuecomment-1494482919

https://github.com/AN3223/dotfiles/commit/b971e899fb155c21a31f1b67200161f99445fb28

guided_lgc : wrong color conversion

nlmeans : would make mpv freezing when using gpu-api=d3d11

[AN3223]

Yeah the color is certainly washed out in your images, and this is a problem I have seen with guided_lgc before. I'm not sure what the cause is. Right now I am working on adding support for testing upscalers in shader_test and guided_lgc is one of the things I plan to test, so if the problem is bad parameters then that should hopefully be fixed soon. Guided filter and the LGC filters are somewhat primitive right now, there are many improvements that can be made.

As for freezing under d3d11, I'm on linux and I have no way of testing this. Is this a problem that only started happening with recent commits?

[AN3223]

By the way, what GPU are you using? And do you still experience freezing when you run nlmeans_lq.glsl?

[hooke007]

Is this a problem that only started happening with recent commits?

After a long time test, I found the commit https://github.com/AN3223/dotfiles/commit/b0612b3d52c72f8af892a286dd19fa6a5facddf4 broke the d3d11's working.

By the way, what GPU are you using

Nvidia ; No freezing with nlmeans_lq

[AN3223]

Ah okay, thank you for your diligent testing! It appears rotations+reflections are stressing your GPU then (usually they have no impact on speed). For a long term solution I will document this problem and build LQ variants of everything. But for now you can either use a different gpu-api or set RI=0 and RFI=0 in your shader files. Let me know if you have any further issues!

[hooke007]

It appears rotations+reflections are stressing your GPU then (usually they have no impact on speed)

My GPU is 4080...

But for now you can either use a different gpu-api

Yeah I know I could make it work with vulkan. Considering d3d11 has the best performance (much greater than opengl and vulkan) for NV&windows, I have to use d3d11 now.

[AN3223]

It appears rotations+reflections are stressing your GPU then (usually they have no impact on speed)

My GPU is 4080...

Yeah rotations+reflections should work well even on a much weaker GPU. I have an RX 570. Something is bugged, but I don't think it's my fault and I don't think I can do anything about it, besides workarounds.

But for now you can either use a different gpu-api

Yeah I know I could make it work with vulkan. Considering d3d11 has the best performance (much greater than opengl and vulkan) for NV&windows, I have to use d3d11 now.

That makes sense. If you're determined to use d3d11 then you can set RI and RFI to zero, or wait until I push out some more LQ shaders.

[hooke007]

Another issue: tested with vulkan. nlmeans seems to makes edge blur?

[AN3223]

Hm. What resolution is this content? Does it help to turn S down?

[hooke007]

What resolution is this content

Any contents I tested (1080p/720p) would share this bug. only lq works correctly.

Does it help to turn S down?

Yes, I could set #define S 1.0 to make it looks well.

[AN3223]

Huh. You haven't changed any other settings besides S?

[hooke007]

Nope.

[AN3223]

I really have no idea. Maybe try WD=0 with S=20?

[hooke007]

Not good.

[AN3223]

WD=1?

[hooke007]

Seems no diff.

[AN3223]

Huh. Try each of these individually to see if any of them fix it: RF=0, RI=0, RFI=0, R=3

[hooke007]

RF=0, RI=0, RFI=0, R=3

X , X , X , X

R=3 still has a little bit blur. The other options seems no obvious changes.

[AN3223]

Hm. Try RI=0 and RFI=0 at the same time. If that doesn't work, try NG=1.

[hooke007]

Try RI=0 and RFI=0 at the same time

No help.

try NG=1

NG=1 only ; RI=0 RFI=0 NG=1 Doesn't work neither.

[AN3223]

I have no clue, that is very strange. For me the blur on 1080p anime is hardly noticeable, for most scenes I have to zoom in to know that NLM is doing anything. I guess you should set S to a comfortable value, and later I will turn the default S value down anyway (the current S value is based on test results that didn't account for EP, so the correct S value should be lower but I need to retest to figure out what the correct value should be, and then adjust sharpening settings).

[hooke007]

Hmmm, more findings here.

I had a mbp2019, nlmeans cannot work. But the only choice of api for Mac is opengl). So I have no way to check if it was nv's driver bug. (That's the only one amd gpu device I had)

For Windows, I also tested with opengl. As you can see, the output is quite different from vulkan.(much less blurring) However, I had never seen anyone tried to use opengl on Windows...

Probably marking it incompatible with Nvdia temporarily in the Doc.

[AN3223]

Wow, thank you for all of this testing! So NLM works fine on Nvidia+opengl, works bad on Nvidia+vulkan, rotations/reflections break Nvidia+d3d11, and it doesn't work on a 2019 Macbook Pro for some reason? (Do other shaders work on Mac?) Now that I think about it, an Nvidia user on Reddit a while back reported that gpu-api=vulkan gave them a black screen, so I guess Nvidia problems are not new. I'll add a warning to the README.

Also what vo are you using?

[hooke007]

So NLM works fine on Nvidia+opengl

It looks like no obvious issues. I'm not sure if it really works well as your design.

it doesn't work on a 2019 Macbook Pro for some reason? (Do other shaders work on Mac?)

It just stop to render the next frame. I have no idea. (Yes)

Also what vo are you using?

gpu

[hooke007]

BTW, gpu-api=opengl would try angle (Direct3D11 through the OpenGL ES translation layer ANGLE) first. Which is not recommended for Windows' users.

We usuallu set gpu-context=win to make sure mpv would use opengl on Windows.

[AN3223]

So NLM works fine on Nvidia+opengl

It looks like no obvious issues. I'm not sure if it really works well as your design.

Soon I will make a Makefile that will make and maintain a set of small lossless images for the README, be on the lookout for that.

it doesn't work on a 2019 Macbook Pro for some reason? (Do other shaders work on Mac?)

It just stop to render the next frame. I have no idea. (Yes)

Huh okay, I will add that to the README too.

Also what vo are you using?

gpu

Is gpu-next any help?

BTW, gpu-api=opengl would try angle (Direct3D11 through the OpenGL ES translation layer ANGLE) first. Which is not recommended for Windows' users.

We usuallu set gpu-context=win to make sure mpv would use opengl on Windows.

Huh okay, I'll add that to the README too.

[hooke007]

• gpu-next with opengl. Seems good.

• d3d11 : freezing.

• vulkan : blurring too.

[AN3223]

Is gpu-next still broken with vulkan though?

[AN3223]

Ah okay

[AN3223]

I updated the README, let me know if there's any issue with it

[hooke007]

LGTM

[AN3223]

Awesome. I'll leave this issue open, I'll close it if/when the guided LGC issue is fixed. As always, feel free to let me know if you have any other questions or issues!

[AN3223]

I just updated the defaults for a bunch of shaders including Guided LGC. There are still many improvements to be made, but at least the LGC shaders should be much better now. Let me know how it works for you.

[hooke007]

guided_lgc seem to be fixed. But nlmeans get more terrible with vulkan —— obvious chroma shifting & greenish output (opengl works fine as always)

[AN3223]

Huh. Weird. Are you sure WD=0 doesn't help?

[hooke007]

I haven't tried to change the param. Tested WD=0 : no improvement

[AN3223]

Weird. I only changed the default settings, mainly WDT, SW, and S. I don't think I modified any actual shader code in the past few days. I think vulkan on nvidia is just hopelessly broken then.

[hooke007]

I remembered that there is an issue about nv's vulkan problem (shader) https://github.com/mpv-player/mpv/issues/9543#issuecomment-1002430902 .

[AN3223]

I remembered that there is an issue about nv's vulkan problem (shader) mpv-player/mpv#9543 (comment) .

Oh interesting. I use pow in a lot of places, so I'm not sure if it's one or many calls to pow that could be the problem. Could you try setting EP=0 and see if it helps? That would avoid one of the calls to pow. I'll also try to refactor other code to avoid pow, we'll see if that helps.

[hooke007]

Could you try setting EP=0 and see if it helps

Unfortunatelly, no.

edit: https://github.com/AN3223/dotfiles/compare/b971e899fb155c21a31f1b67200161f99445fb28...4714641bbda908f10188335661f95f6aff575be8 guided_lgc : #define E 100.0 fix the chroma nlmeans : #define S 5.0 generated more greenish output

[AN3223]

Okay, right now I'm running some tests, I'll have a patch soon after that.

[AN3223]

Try now?

[hooke007]

Confirmed that blurring and chroma shifting in vulkan was fixed.

[AN3223]

Awesome! It even works with EP=1? Does nlmeans_sharpen_denoise.glsl work too?

[hooke007]

It even works with EP=1

Yes, I usually don't touch internal param. And nlmeans_sharpen_denoise works too.

P.s. About d3d11's freezing : If forgetting to close it, it seems to plan to eat up the RAM :)

[AN3223]

It even works with EP=1

Yes, I usually don't touch internal param. And nlmeans_sharpen_denoise works too.

Awesome, those are pretty much the only places where nlmeans still calls pow, I'll update the README. Thank you for your diligent testing and your advice!

P.s. About d3d11's freezing : If forgetting to close it, it seems to plan to eat up the RAM :)

Hm, that's unexpected, but I still have no idea.

[hooke007]

ref https://github.com/AN3223/dotfiles/issues/6#issuecomment-1494722776 I tested gpu-context=angle (The fisrt auto-choice of gpu-api=opengl). No luck, nlmeans was broken. Even though setting RI, RFI = 0, it could not be fixed.

[AN3223]

Oh okay, I guess I'll add this to the README too