[Question] Is there better options for image denoising?

[XCanG]

Right now I mostly stick with this setup, as it looks best out of 6 presets coming from readme:

shaders="~~/shaders/Anime4K_Denoise_Bilateral_Mode.glsl;~~/shaders/Anime4K_DarkLines_HQ.glsl;~~/shaders/Anime4K_ThinLines_HQ.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur.glsl"

however I have artifacts with it in some places. I figured out, that the reason of it Anime4K_Denoise_Bilateral_Mode.glsl. Here is an example of this artifact: basically it was a subtitles in opening, which become noisy and ruin picture.

If there are better alternatives, I would rather try them. I have pretty good components which should be able to handle slower algorithms.

If you need settings, here is related to render:

profile=gpu-hq
vo=gpu
hwdec=nvdec-copy
hwdec-codecs=all
gpu-api=vulkan

deband=yes
deband-iterations=4
deband-threshold=82
deband-range=16
deband-grain=24

scale=ewa_lanczossharp
cscale=ewa_lanczossharp

glsl-shaders="~~/shaders/Anime4K_Denoise_Bilateral_Mode.glsl;~~/shaders/Anime4K_DarkLines_HQ.glsl;~~/shaders/Anime4K_ThinLines_HQ.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur.glsl"

[Jules-A]

I use Anime4K_Denoise_Bilateral_Median.glsl and change INTENSITY_SIGMA to 0.15. It's quite a bit better than mode but is heavier and needs to be used with at least Anime4K_Upscale_CNN_M_x2_Deblur to counteract the blurring. It unfortunately is nowhere near as good on text as MadVR but the only other better denoiser is the heavy which is so strong it destroys image quality on normal videos and the upscaled (Anime4K_Upscale_CNN_M_x2_Denoise) ones but they are too heavy for my system.

[XCanG]

So, Anime4K_Upscale_CNN_M_x2_Denoise may be better, but slow, then Anime4K_Denoise_Bilateral_Median.glsl alternative, but have artifacts in different place? And how slow is MadVR? Or else how much it can destruct image? Are there any benefit from it?

[XCanG]

Hm, with Anime4K_Upscale_CNN_M_x2_Denoise it definitely better: I, however can't say anything about performance impact, as statistics on I all zeroes on Vulkan.

[Jules-A]

So, Anime4K_Upscale_CNN_M_x2_Denoise may be better, but slow, then Anime4K_Denoise_Bilateral_Median.glsl alternative, but have artifacts in different place? And how slow is MadVR? Or else how much it can destruct image? Are there any benefit from it?

MadVR can't be used in MPV and newer Anime4K shaders can't be used with it either since Anime4K is GLSL only now. MadVR has many different ways to configure but currently I use only use it for 480/720p, it has it's draw backs like adding a significant amount of blur to the image but it's far better than a tonne of noise and isn't very expensive either. I actually tested my settings and it looks like line thinning/darkening is making the text worse, rather than the denoiser.

[XCanG]

What about other Denoise files? I see several, but not sure how can I distinguish them: Anime4K_Upscale_CNN_L_x2_Denoise.glsl Anime4K_Upscale_CNN_M_x2_Denoise.glsl Anime4K_Upscale_CNN_UL_x2_Denoise.glsl

[XCanG]

I actually tested my settings and it looks like line thinning/darkening is making the text worse, rather than the denoiser.

This is true, but it work in other places, I don't see if I can replace it to something else as it work both with borders of characters and text. I just think that without AI you can't distinguish them.

[Jules-A]

What about other Denoise files? I see several, but not sure how can I distinguish them: Anime4K_Upscale_CNN_L_x2_Denoise.glsl Anime4K_Upscale_CNN_M_x2_Denoise.glsl Anime4K_Upscale_CNN_UL_x2_Denoise.glsl

They are pretty much the same except L and UL are heavier on the system and slightly better quality.

[XCanG]

Hm, with Anime4K_Upscale_CNN_M_x2_Denoise it definitely better:

Oh, wait, I just found what it not appearing in shaders list, it looks like it can't load or something.

I don't see any errors in console and any position showed in I statistics.

[Jules-A]

Hm, with Anime4K_Upscale_CNN_M_x2_Denoise it definitely better:

Oh, wait, I just found what it not appearing in shaders list, it looks like it can't load or something.

I think maybe you forgot .glsl, I forgot to add it after copying and pasting from Windows (which doesn't include the extension).

[XCanG]

I think maybe you forgot .glsl, I forgot to add it after copying and pasting from Windows (which doesn't include the extension).

no, I set this binds:

CTRL+7 no-osd change-list glsl-shaders set "~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DarkLines_HQ.glsl;~~/shaders/Anime4K_ThinLines_HQ.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur.glsl"; show-text "Anime4k: 1080p (Perceptual Quality, CNN M Denoise)"
CTRL+8 no-osd change-list glsl-shaders set "~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_DarkLines_HQ.glsl;~~/shaders/Anime4K_ThinLines_HQ.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur.glsl"; show-text "Anime4k: 1080p (Perceptual Quality, CNN L Denoise)"
CTRL+9 no-osd change-list glsl-shaders set "~~/shaders/Anime4K_Upscale_CNN_UL_x2_Denoise.glsl;~~/shaders/Anime4K_DarkLines_HQ.glsl;~~/shaders/Anime4K_ThinLines_HQ.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur.glsl"; show-text "Anime4k: 1080p (Perceptual Quality, CNN UL Denoise)"

All files in place (and I copy it's name, so there is no typo).

[XCanG]

There is output on all CTRL+7 to 9:

This with Ctrl+5:

[XCanG]

I try to switch and disable some settings, like Vulkan, however it still can't load for whatever reason... Console is also clear.

[Jules-A]

It may be because you are using 1080p monitor with 1080p content. Because of this line:

//!WHEN OUTPUT.w NATIVE.w / 1.200 > OUTPUT.h NATIVE.h / 1.200 > *

It only upscales when the display resolution is 1.2x the height or (?) width of the content you are trying to watch. On my 1440p monitor, when I try watching in a smaller window with 1080p content, the shaders won't load. You can try editing that line but I'm not sure what the consequences are.

[XCanG]

Oh, I really have 1080p, but I can use DXR for make it 4k. Let me try it.

[XCanG]

Yes, now it work. And yes I don't see at this moment any artifacts at the text, at least. However why it not denoising when requirement for upscale is not met? It is something that algorithm heavily related on something or just synthetical limit?

[XCanG]

And now I see in task manager how it affect. Both M and L are not really that heavy, but UL starting to produce lags, however I can also say, what it took only 3 cores to 100%, so it probably not fully parallel process. I don't see any spikes on GPU, so I assume it use CPU for denoising.

[TianZerL]

it use CPU for denoising.

GLSL shader only runs in GPU.

why it not denoising when requirement for upscale is not met?

Just change 1.200 in the source to 0.9 or lower to keep the denoising shader running anytime.

[XCanG]

it use CPU for denoising.

GLSL shader only runs in GPU.

Hm, I only see 20% GPU usage and 3 cores to 100%.

[TianZerL]

Hm, I only see 20% GPU usage and 3 cores to 100%.

What's your CPU?

CPU only does scheduling tasks of shaders. The more shaders you load, the higher usage CPU get. Many layers of shaders are integrated in a single file of the UL shader.

[XCanG]

Ryzen 9 3900X

[bloc97]

Look forward to the v3.2 update, as it will incorporate a major overhaul to the denoising algorithms. However this update will be delayed as my personal training computer died and I won't use university resources for a personal extracurricular project. New GPUs are also very much inexistent right now...... I can't promise a date, but I think Q1 2021 would be reasonable.

[XCanG]

I'll set this repository for watch and will wait.

[Artins90]

I'll set this repository for watch and will wait.

I did a quick test using the current 4.0 RC with the following shaders (my shader configuration might not be optimal): glsl-shaders="~~/shaders/Anime4K_Restore_CNN_Light_M.glsl;~~/shaders/Anime4K_Upscale_Denoise_CNN_x2_L.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Restore_CNN_Moderate_L.glsl;~~/shaders/Anime4K_Upscale_Denoise_CNN_x2_M.glsl;"

https://imgsli.com/NjUwNjU

At native 4K it looks great and the noise is basically gone when viewing in full-screen. Really looking forward to the final release and to further tweaks/options to the shader combinations.

[bloc97]

The new shaders are much simpler/easier to use. For 1080p->4K it is simply: ~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_Moderate_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_L.glsl Here is what I get: https://imgsli.com/NjUwNzU

I have updated the optimized GLSL instructions for 4.x.

[Jules-A]

The new shaders are much simpler/easier to use. For 1080p->4K it is simply: ~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_Moderate_VL.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_L.glsl Here is what I get: https://imgsli.com/NjUwNzU

I have updated the optimized GLSL instructions for 4.x.

That doesn't really show the effectiveness of denoising though? How is the speed vs the old Bilateral_Median?

The new shaders are a little puzzling for me, they seem to reduce lines a bit too much for my liking even though I previously used line thinning with 0.36 Strength and 2 iterations along with slightly reduced strength line darkening. Do I forgo those shaders now and edit something in another file?

[bloc97]

I can try to explain what the shader does. The "thinning lines" effect is actually an illusion. The shader does not thin lines at all. It recovers original lines from badly resampled lines which often happen when digitally compositing anime.

Consider this test pattern, brush strokes of size 1 to 10 from left to right:

We can artificially degrade it by downscaling the image by 40% (1080p->648p) then upscale back to 1080p using bicubic. Then we apply JPEG compression of 50%, which is similar to moderately compressed anime seen online (streams/youtube). If you look at the brush strokes of size 1 and 2, they look much larger now due to the nonlinear degradation we caused by resampling and compressing the image.

Then we pass it through the line reconstruction shader (moderate, VL variant):

It now looks almost exactly like the original image, without noise and bicubic degradation. Best viewed zoomed in, they are 1080p images.

[bloc97]

Another interesting experiment is to try on different amounts of degradation for the 1 and 2 width strokes. HR Image:

JPEG compression of 50%

After the line reconstruction shader (moderate, VL variant): Most of the lines have the same thickness, but 100% and 90% lines are slightly (less than 0.2px) thinned as they are outliers.

However, anime will almost never have these two cases except for text overlays, baked subtitles and high quality background images. (Try to find these edge cases in modern 1080p anime, I tried and could not... Could be proven wrong.) Furthermore, the edge cases are still very hard for the relatively small network to learn. You would need advanced architectures with millions of parameters, which makes real time out of the question.

[Jules-A]

That's pretty cool info but I don't really care too much about what the shader is really doing as long as the end result is good and I was really happy with how those shaders looked on 720/1080p decently encoded anime on anything that wasn't built-in text. It did however make low res/poorly encoded videos unwatchable though.

Also... Umm... you didn't answer any of my questions :/

EDIT: I also noticed there's a decent bit of aliasing added with new line reconstruction shader, what's the best way to avoid this?

[bloc97]

@Jules-A currently the new shaders don't work well with older anime, I still need to collect more data. Do you know which anime are problematic? Some screenshots of original low res images would be very appreciated. Also most of the older shaders are still going to be supported except a few like deblur-cnn, you can still use them, is there one shader that you used that I removed?

The aliasing is still being worked on, if you have some screenshots of cases with severe aliasing (before applying shaders) that I can put in the validation set that would be of big help!

[Jules-A]

@Jules-A currently the new shaders don't work well with older anime, I still need to collect more data. Do you know which anime are problematic? Some screenshots of original low res images would be very appreciated. Also most of the older shaders are still going to be supported except a few like deblur-cnn, you can still use them, is there one shader that you used that I removed?

The aliasing is still being worked on, if you have some screenshots of cases with severe aliasing (before applying shaders) that I can put in the validation set that would be of big help!

Pretty much any older anime with a 480p base resolution. I only have Honey and Clover (downloaded straight from CR) but here's a comparison of my current shaders: https://i.imgur.com/9FRbNR2.png and some lowish settings in MadVR: https://i.imgur.com/PUNfkV9.png . I've had far worse problems with other old anime but I remember having major issues in Le Chevalier D'Eon from Funimation (the show also had a lot of noise that the billinear denoisers weren't good enough).

You can probably see that in the screenshots I sent, my Anime4k shaders murder the background turning it in to some sort of painting with awful aliasing and lines become all thick and weird with aliasing nowhere close to the actual line.

As for aliasing before shaders, I'm really not sure, they just seem to get severely aliased by the shaders rather than starting out aliased.

The shaders I'm currently using: glsl-shaders="~~/shaders/Anime4K_Denoise_Bilateral_Median2.glsl;~~/shaders/Anime4K_DarkLines_HQ2.glsl;~~/shaders/Anime4K_ThinLines_HQ2.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Deblur2.glsl" with 0.16 Str median, above changed to lines and slight decrease to deblur str and threshold.

[bloc97]

I though we were talking about the new line reconstruction shaders? Here's what I get on a few 480p anime with ~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_Light_M.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl;

As I don't have the originals, I went on Funimation and CR to take random screenshots of the two anime you mentioned. Here's a comparison using the above shaders, it's not great but not unwatchable either... https://imgsli.com/NjUzMTI https://imgsli.com/NjUzMTU

I also only have a few "older" 480p anime. Monster: https://imgsli.com/NjUzMTg Death Note: https://imgsli.com/NjUzMTc Zero no Tsukaima: https://imgsli.com/NjUzMjA Zipang: https://imgsli.com/NjUzMTk

Some overshoot (over sharpening) can be seen but it can be fixed by using the soft versions of restore instead (in the repo, not yet released).

[Artins90]

The current version does add some slight softness to backgrounds and textures in modern 1080p anime (you might need a 4k screen to notice it or zooming). For example the railing in this comparison is a bit blurrier than in the source: https://imgsli.com/NjUzMzU Between the two, personally I still prefer Anime4k since in it's current state the pros far outweigh the small softness added to backgrounds.

[Jules-A]

I though we were talking about the new line reconstruction shaders? Some overshoot (over sharpening) can be seen but it can be fixed by using the soft versions of restore instead (in the repo, not yet released).

Oh sorry, I haven't had time to properly play around with the new shaders so I'm just using the old 3.1 shaders as known good for 720/1080p anime. I just quickly tested 4.0 and the aliasing is pretty noticeable (you can see it even in Artins comparison). It's probably not noticeable or annoying for most people but I'm very sensitive to aliasing for some reason.

EDIT: I still haven't got an answer about the denoising from my first post since 4.0 (which this issue is about lol). I know I can test it myself but I was hoping to get your opinion on it?

Also it does look like the new shaders are far better than the ones I'm currently using for older anime.

EDIT2: The new shaders, especially with the addition of soft are miles better on older low res anime but whatever I do I can't seem to get 720p/1080p anime close in quality on my 1440p monitor, it seems like there's not as much deblurring with the restore shaders, even without using the soft variant. I'm really not too sure what I can/should run with my rx570 (the presets haven't been too helpful so far).