A Proposal for Inferencing High-Resolution Images with limited gpu vram less than 6GB.

[alexaex]

We can split the high-resolution image into multiple fixed size patches without overlap, then do inference on each patch, and finally merge the upscaled patches to obtain the full high-resolution image. I have already implemented this, and it is indeed feasible for enabling low VRAM GPUs like RTX3060 Laptop with 6GB VRM to upscale 1080P images. Notably, it seems to have no apparent negative effect on the quality of the upscaled image. The motivation from vision transformer and your paper, in vision transformer the image is split into multiple patches for tokenization, and in your paper actually train proportion of high resolution image instead of the whole image. Moreover, I suppose this apporach can also work for accelerating inference with multiple GPUs.

[Kiteretsu77]

Great Idea! Do you want to take a look to my previous repo: https://github.com/Kiteretsu77/FAST_Anime_VSR. I have tested it to be functionable in 8GB 3060Ti for video Super-Resolution inference. It also use division to high resolution images, and also with TensorRT acceleration. However, this repo is hard to be installed.