search

XPixelGroup / BasicSR

Open Source Image and Video Restoration Toolbox for Super-resolution, Denoise, Deblurring, etc. Currently, it includes EDSR, RCAN, SRResNet, SRGAN, ESRGAN, EDVR, BasicVSR, SwinIR, ECBSR, etc. Also support StyleGAN2, DFDNet.
https://basicsr.readthedocs.io/en/latest/
Apache License 2.0
6.52k stars 1.15k forks source link

Can the code be modified to accept multiple LR images for a single HR output image? #198

Open TomArrow opened 4 years ago

TomArrow commented 4 years ago

I have a scenario where I have multiple low resolution scans from the same film, but each is slightly different from the next. I'd like to train the algorithm to look at all the different LR images (with the identical alignment ofc) and use that information to generate one high resolution image. How could I do this?

I was thinking about "tricking" the algorithm a little by just mixing up/interleaving the pixels, but it doesn't seem like a great solution to me and might not give the ideal results due to the random 128x128 blocks cropped out of the image for training.

So ideally I would just have 4 folders. LR1, LR2, LR3, HR and the images in each folder would correspond to the other folders by filename, like it is now.

The reason why I want this is because I think that the different images might give the algorithm more clues on how to reconstruct the full image.

xinntao commented 4 years ago

What you described is video super-resolution or multi-frame super-resolution. Have a look at our EDVR framework for Video SR. https://github.com/xinntao/EDVR

TomArrow commented 4 years ago

That looks awesome, thanks, I will have a look.