k4yt3x / video2x

A lossless video/GIF/image upscaler achieved with waifu2x, Anime4K, SRMD and RealSR. Started in Hack the Valley II, 2018.
https://video2x.org
GNU Affero General Public License v3.0
9.82k stars 955 forks source link

Using GPU #426

Closed Kameechewa closed 3 years ago

Kameechewa commented 3 years ago

Hello, I just found this tool tonight and don't really have any clue what I am doing. I have successfully started upscaling a 22 minute, 41,029 frame video, and it's jumping around from 14 hours to 22 minutes remaining. I have an i7-7700k with an EVGA RTX 2070 Super. For the first 2 minutes my GPU utilization is zero and my CPU is pegged at 100% on all cores. Then it starts processing frames and the CPU utilization drops to around 10% while the GPU is at 20%-30%. My question is, does this sound normal? I have changed the number of processes to 4 (someone please explain what that does better) and that is all. I am using the Waifu2X NCNN Vulkan Driver. I know Vulkan has something to do with graphics but that's about it. Is it normal for it to peg the CPU like that in the beginning? Is there a reason that only 20%-30% of my GPU is being utilized? Can I increase the utlization/speed of the upscale?

Update 1: I stopped the previous run and started again with 16 processes. It took about 4 minutes to start this time and my VRAM is almost completely used versus it being about 66% previously. But now my CPU is staying pegged at 100% and my GPU is only at 2%. That doesn't seem right. However, the upscale is now taking minutes with a FPS of about 92-100. But the Frame Preview window is blank so that is worrisome.

Update 2: It's been finished for about 8 minutes and nothing is happening. It's just sitting here at 100% and the FPS are slowly dropping as is CPU usage. I have decided to go back and run with the 4 processes and let it go all night and hope it's successful in the morning. At least now the Frame Preview is working again. Also, changing the view in Task Manager to CUDA shows 0%, but if this is Vulkan maybe that's why? Anyways, changing it to compute1 shows 100% utilization, if that means anything.

Update 3: It did complete successfully however the image quality is not as improved as I'd hoped. The source is only 480p so I am now running at 4x with 6 processes.

k4yt3x commented 3 years ago

To understand the initial CPU utilization spike, you'll first understand how v2x works. v2x first extracts all frames from the video before starting to process them with waifu2x/srmd/etc. The first step of extracting the frames uses CPU, hence the CPU usage spike.

ncnn-Vulkan-based drivers do not use CUDA. waifu2x-caffe uses CUDA/cuDNN. This is why do don't see any CUDA utilization rates.

It is not recommended to use multi-processing for machine-learning based drivers. Use only one process unless you're running Anime4KCPP. It's better to increase the crop/tile size for machine-learning-based drivers. In waifu2x-ncnn-vulkan's case, it will be the tile size. This value defaults to 400. You can increase it until the VRAM required exceeds what your card can provide.

You can also tune waifu2x-ncnn-vulkan's parameters like changing de-noise level to 3 from 2. You may also want to give waifu2x-caffe a try. I've noticed it producing outputs with less artifacts.

Kameechewa commented 3 years ago

Ah ok, thank you for the explanation. Is there somewhere that I can read up on the differences between them and what settings do what?

k4yt3x commented 3 years ago

There's a driver guide but it's outdated: https://github.com/K4YT3X/video2x/wiki/Drivers I'll need to update it later. If you want to get the highest-quality output, use waifu2x-caffe if you have an anime-like video, use realsr if you have a real-life video.