search

mafiosnik777 / enhancr

Video Frame Interpolation & Super Resolution using NVIDIA's TensorRT & Tencent's NCNN inference, beautifully crafted and packaged into a single app
https://dsc.gg/enhancr
GNU General Public License v3.0
534 stars 29 forks source link

Support Real-ESRGAN x2 Plus #44

Open x5nder opened 3 months ago

x5nder commented 3 months ago

I've run a bunch of tests on upscaling SD movies (VHS and DVD) and found that using that using Real-ESRGAN x2 Plus (RealESRGAN_x2plus.pth) is offering much better quality than the included RealESRGAN encoder, which tends to smooth out details so people look like they're made of plastic.

Unfortunately, using x2 Plus as custom model has a severe impact on the transcoding speed (going from approx 60-110fps for RealESRGAN TensorRT 2x to around 13-24fps for the custom model converted to TensorRT engine).

Is there any way to speed this up, or is there any way you could implement a fast x2 (or x4) upscaler that is much better for realistic movies (as opposed to anime content)?

styler00dollar commented 3 months ago

RealESRGAN_x2plus.pth is ESRGAN (RRDBNet) which is known to be slow while the included model is also from the RealESRGAN repository but based on SRVGGNetCompact which is much smaller. That are completely different architectures. Behavior is expected and there isn't really something you can do. You would need to find a different model to make it faster.