search

open-mmlab / mmagic

OpenMMLab Multimodal Advanced, Generative, and Intelligent Creation Toolbox. Unlock the magic 🪄: Generative-AI (AIGC), easy-to-use APIs, awsome model zoo, diffusion models, for text-to-image generation, image/video restoration/enhancement, etc.
https://mmagic.readthedocs.io/en/latest/
Apache License 2.0
6.85k stars 1.05k forks source link

Apply second-order degradation of Real-ESRGAN to video-based model #715

Closed zzh-tech closed 2 years ago

zzh-tech commented 2 years ago

Can you provide any instructions (configs) to apply second-order degradation of Real-ESRGAN to video-based model like EDVR? Thanks a lot!

ckkelvinchan commented 2 years ago

Hello, you can refer to the configuration file of RealBasicVSR. It applies second-order degradation model to train a video network.

zzh-tech commented 2 years ago

Hi, thanks! EDVR cannot handle the case of MultipleGT. https://github.com/open-mmlab/mmediting/blob/4116124f390d82d0d7d2bacfa847c1ab602f6996/configs/restorers/real_basicvsr/realbasicvsr_wogan_c64b20_2x30x8_lr1e-4_300k_reds.py#L31 Is there a way to modify the pipeline directly to retain a GT image without modifying the EDVR code?

ckkelvinchan commented 2 years ago

Hi, thanks! EDVR cannot handle the case of MultipleGT.

https://github.com/open-mmlab/mmediting/blob/4116124f390d82d0d7d2bacfa847c1ab602f6996/configs/restorers/real_basicvsr/realbasicvsr_wogan_c64b20_2x30x8_lr1e-4_300k_reds.py#L31

Is there a way to modify the pipeline directly to retain a GT image without modifying the EDVR code?

I am afraid we cannot support this without changing the code, since the LR images are generated from GT images. Therefore, in order to have multiple LR, we need to load multiple GT in advance.

Nevertheless, you can easily modify the code in the EDVR model to select the center GT (You need to modify the forward function accordingly).

zzh-tech commented 2 years ago

Got it, thanks!

zmlmanly commented 2 years ago

Hi, thanks! EDVR cannot handle the case of MultipleGT. https://github.com/open-mmlab/mmediting/blob/4116124f390d82d0d7d2bacfa847c1ab602f6996/configs/restorers/real_basicvsr/realbasicvsr_wogan_c64b20_2x30x8_lr1e-4_300k_reds.py#L31

Is there a way to modify the pipeline directly to retain a GT image without modifying the EDVR code?

I am afraid we cannot support this without changing the code, since the LR images are generated from GT images. Therefore, in order to have multiple LR, we need to load multiple GT in advance.

Nevertheless, you can easily modify the code in the EDVR model to select the center GT (You need to modify the forward function accordingly).

@ckkelvinchan , Hi. How to save the LR images by generating with second-order degradation of Real-ESRGAN when train the RealBasicVSR based REDS dataset. I am training the 2x SR model. I want to get the LR images based REDS dataset. Thanks.