bloc97 / Anime4K

A High-Quality Real Time Upscaler for Anime Video
https://bloc97.github.io/Anime4K/
MIT License
18k stars 1.34k forks source link

[Feature Request] A shader that's in between the normal and soft restore shaders. #133

Open Jules-A opened 2 years ago

Jules-A commented 2 years ago

Is your feature request related to a problem? Please describe. I've been experimenting with the new shaders but I'm not really happy with the restore shaders. The normal shader introduces a massive amount of aliasing and the soft shader removes a lot of detail from the image.

Describe the solution you'd like I'd like a restore shader that's not too aliased and doesn't kill too much detail, something in between the normal and soft restore shaders.

Describe alternatives you've considered Right now I've been trying to get around it by using this combination of shaders: "~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Denoise_Bilateral_Median3.glsl;~~/shaders/Anime4K_Restore_CNN_S.glsl;~~/shaders/Anime4K_DarkLines_HQ3.glsl;~~/shaders/Anime4K_ThinLines_HQ3.glsl;~~/shaders/Anime4K_Restore_CNN_Soft_L.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_L.glsl" Which produces the best image quality without lagging (imo, I'm probably more sensitive to aliasing than most people) but loses a decent amount of detail on fine objects and turns some stuff that shouldn't be into lines. I also haven't had any success with the new denoise shaders or the new line dark/thinners (they produce massive amounts of lagg) so I'm using the old 3.1 ones.

The modifications to the shaders are: DarkLines: STRENGTH 1.30 ThinLines: STRENGTH 0.12, ITERATIONS 2 Billateral_Median: INTENSITY_SIGMA 0.13, SPATIAL_SIGMA 1.05 (not sure if that does anything).

Additional context Using an RX570 on a 27" 1440p 75hz monitor with interpolation enabled in windowed 2560x1400.

Screenshots (comparing HQ even if I can't run since LQ all look terrible) using 1080p Funimation source: No shaders My shaders Mode A (HQ) Mode B (HQ) Mode C (HQ) Mode A+A (HQ) Mode B+B (HQ) Mode C+A (HQ)

bloc97 commented 2 years ago

There was Anime4K_Restore_CNN_Light_Soft shaders in the repo before the v4.0.1 release, do they look like what you request? https://github.com/bloc97/Anime4K/tree/6894d0a266df333920f23f1f61caec44f5136da1/glsl/Restore

Edit: The reason for their removal was that the difference betwen light and moderate was negligible and it would be a nightmare to support two dozen shaders for extremely rare use cases...

Jules-A commented 2 years ago

There was Anime4K_Restore_CNN_Light_Soft shaders in the repo before the v4.0.1 release, do they look like what you request? https://github.com/bloc97/Anime4K/tree/6894d0a266df333920f23f1f61caec44f5136da1/glsl/Restore

Edit: The reason for their removal was that the difference betwen light and moderate was negligible and it would be a nightmare to support two dozen shaders for extremely rare use cases...

I tried them before but in my previous attempts it left the image too blurry. I think I will try a mix of the old shaders with the new to see if anything changes.

bloc97 commented 2 years ago

I tried them before but in my previous attempts it left the image too blurry. I think I will try a mix of the old shaders with the new to see if anything changes.

That's what I figured too when testing the light shaders. Either its too blurry or it enhances noise. There's a very delicate balance when doing noise reduction, and the shaders currently err on the side of caution (make sure no noise is amplified). The side effect is that sometimes textures are blurred out... This is inevitable as the SNR can drop below 0dB (more noise than signal) for very light textures in anime when compressed using MPEG.

I'm also currently doing research on a quick way of isolating lines, textures and flat shading regions for better image reconstruction. Using a very deep U-Net like those in SRGANs or newer waifu2x versions is out of the question as they will definitively not run in real time, even with two 3090s...

wolfdaddyh commented 2 years ago

After trying many possible combos, I found it’s best to put the restore shader after upscale shader to reduce artifact. My monitor is 1080p and I use upscale x2-> restore > downscale x4 -> upscale x2-> restore -> downscale x2. It won’t be as sharp but a lot less ringing

Jules-A commented 2 years ago

After trying many possible combos, I found it’s best to put the restore shader after upscale shader to reduce artifact. My monitor is 1080p and I use upscale x2-> restore > downscale x4 -> upscale x2-> restore -> downscale x2. It won’t be as sharp but a lot less ringing

I can't quite remember but I think doing the restores after upscaling is pretty expensive which is why I opted to do everything without upscaling first. It might be a lot easier to do with a 1080p monitor or using S quality restores though.

I'm also currently doing research on a quick way of isolating lines, textures and flat shading regions for better image reconstruction.

Sounds really interesting :)