search

LoSealL / VideoSuperResolution

A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
MIT License
1.61k stars 295 forks source link

Inference of large images #100

Closed Dejan1969 closed 4 years ago

Dejan1969 commented 4 years ago

Inferring large images will cause error :

$python3 eval.py esrgan --cuda --auto_rename -t DSC_0056.JPG 2020-03-08 08:49:08,294 INFO: LICENSE: ESRGAN is implemented by Xintao Wang. @xinntao https://github.com/xinntao/ESRGAN 2020-03-08 08:49:12,067 INFO: Total params: 77223884 2020-03-08 08:49:12,068 WARNING: trying to restore state for optimizer optd, but failed. 2020-03-08 08:49:12,068 WARNING: trying to restore state for optimizer optg, but failed. 2020-03-08 08:49:12,078 INFO: Inferring [dnet][rrdb] at epoch 0 Infer: 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "eval.py", line 117, in main() File "eval.py", line 113, in main t.infer(ld, config) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Framework/Trainer.py", line 207, in infer self.fn_infer_each_step(items) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Framework/Trainer.py", line 227, in fn_infer_eachstep outputs, = self.model.eval([feature]) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Models/Esrgan.py", line 81, in eval sr = self.rrdb(inputs[0]).cpu().detach() File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, kwargs) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Models/esrgan/architecture.py", line 47, in forward x = self.model(x) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, *kwargs) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 100, in forward input = module(input) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, kwargs) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Models/esrgan/block.py", line 87, in forward output = x + self.sub(x) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, kwargs) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 100, in forward input = module(input) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, *kwargs) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Models/esrgan/block.py", line 248, in forward out = self.RDB1(x) File "/home/xxxxxx/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(input, kwargs) File "/home/xxxxxx/VideoSuperResolution/VSR/Backend/Torch/Models/esrgan/block.py", line 221, in forward x2 = self.conv2(torch.cat((x, x1), 1)) RuntimeError: CUDA out of memory. Tried to allocate 5.75 GiB (GPU 0; 10.76 GiB total capacity; 6.30 GiB already allocated; 3.58 GiB free; 6.32 GiB reserved in total by PyTorch)

This issue can be solved with spliting the image into partially overlapping patches and stiching together after processing. You may consider this approach: https://github.com/idealo/image-super-resolution/blob/master/ISR/utils/image_processing.py

https://github.com/idealo/image-super-resolution/issues/14

LoSealL commented 4 years ago

There is a splitting tool in Tools/NtireHelper.py, it's an exp tool so try to read the code and comments. You will use --task=divide and --task=combine to split and re-combine the huge images.

Dejan1969 commented 4 years ago

Thank you for clarifying, I will try it once I get eval.py to work