search

Tencent / Real-SR

Real-World Super-Resolution via Kernel Estimation and Noise Injection
Apache License 2.0
785 stars 118 forks source link

readme

RealSR

logo

Real-World Super-Resolution via Kernel Estimation and Noise Injection (CVPR 2020 Workshops)

Xiaozhong Ji, Yun Cao, Ying Tai, Chengjie Wang, Jilin Li, and Feiyue Huang

Tencent Youtu Lab

Our solution is the winner of CVPR NTIRE 2020 Challenge on Real-World Super-Resolution in both tracks.

(Official PyTorch Implementation)

Update - September 1, 2020

Update - May 26, 2020

Introduction

Recent state-of-the-art super-resolution methods have achieved impressive performance on ideal datasets regardless of blur and noise. However, these methods always fail in real-world image super-resolution, since most of them adopt simple bicubic downsampling from high-quality images to construct Low-Resolution (LR) and High-Resolution (HR) pairs for training which may lose track of frequency-related details. To address this issue, we focus on designing a novel degradation framework for real-world images by estimating various blur kernels as well as real noise distributions. Based on our novel degradation framework, we can acquire LR images sharing a common domain with real-world images. Then, we propose a real-world super-resolution model aiming at better perception. Extensive experiments on synthetic noise data and real-world images demonstrate that our method outperforms the state-of-the-art methods, resulting in lower noise and better visual quality. In addition, our method is the winner of NTIRE 2020 Challenge on both tracks of Real-World Super-Resolution, which significantly outperforms other competitors by large margins.

RealSR

If you are interested in this work, please cite our paper

@InProceedings{Ji_2020_CVPR_Workshops,
               author = {Ji, Xiaozhong and Cao, Yun and Tai, Ying and Wang, Chengjie and Li, Jilin and Huang, Feiyue},
               title = {Real-World Super-Resolution via Kernel Estimation and Noise Injection},
               booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
               month = {June},
               year = {2020}
     }

and challenge report NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results

@article{Lugmayr2020ntire,
        title={NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results},
        author={Andreas Lugmayr, Martin Danelljan, Radu Timofte, Namhyuk Ahn, Dongwoon Bai, Jie Cai, Yun Cao, Junyang Chen, Kaihua Cheng, SeYoung Chun, Wei Deng, Mostafa El-Khamy Chiu, Man Ho, Xiaozhong Ji, Amin Kheradmand, Gwantae Kim, Hanseok Ko, Kanghyu Lee, Jungwon Lee, Hao Li, Ziluan Liu, Zhi-Song Liu, Shuai Liu, Yunhua Lu, Zibo Meng, Pablo Navarrete, Michelini Christian, Micheloni Kalpesh, Prajapati Haoyu, Ren Yong, Hyeok Seo, Wan-Chi Siu, Kyung-Ah Sohn, Ying Tai, Rao Muhammad Umer, Shuangquan Wang, Huibing Wang, Timothy Haoning Wu, Haoning Wu, Biao Yang, Fuzhi Yang, Jaejun Yoo, Tongtong Zhao, Yuanbo Zhou, Haijie Zhuo, Ziyao Zong, Xueyi Zou},
        journal={CVPR Workshops},
        year={2020},
    }

Visual Results

0

1

Quantitative Results Compared with Other Participating Methods

'Impressionism' is our team. Note that the final decision is based on MOS (Mean Opinion Score) and MOR (Mean Opinion Rank).

0

1

Qualitative Results Compared with Other Participating Methods

'Impressionism' is our team.

0

1

Dependencies and Installation

This code is based on BasicSR.

Pre-trained models

Testing

Download dataset from NTIRE 2020 RWSR and unzip it to your path.

For convenient, we provide Corrupted-te-x and DPEDiphone-crop-te-x.

cd ./codes

DF2K: Image processing artifacts

  1. Modify the configuration file options/df2k/test_df2k.yml
    • line 1 : 'name' -- dir name for saving the testing results
    • line 13 : 'dataroot_LR' -- test images dir
    • line 26 : 'pretrain_model_G' -- pre-trained model for testing
  2. Run command : CUDA_VISIBLE_DEVICES=X python3 test.py -opt options/df2k/test_df2k.yml
  3. The output images is saved in '../results/'

DPED: Smartphone images

  1. Modify the configuration file options/dped/test_dped.yml
    • line 1 : 'name' -- dir name for saving the testing results
    • line 13 : 'dataroot_LR' -- test images dir
    • line 26 : 'pretrain_model_G' -- pre-trained model for testing
  2. Run command : CUDA_VISIBLE_DEVICES=X python3 test.py -opt options/dped/test_dped.yml
  3. The output images is saved in '../results/'

Training

Track 1

  1. prepare training data

    • specify dataset paths in './preprocess/path.yml' and create bicubic dataset : python3 ./preprocess/create_bicubic_dataset.py --dataset df2k --artifacts tdsr

    • run the below command to collect high frequency noise from Source : python3 ./preprocess/collect_noise.py --dataset df2k --artifacts tdsr

  2. train SR model

    • Modify the configuration file options/df2k/train_bicubic_noise.yml
    • Run command : CUDA_VISIBLE_DEVICES=4,5,6,7 python3 train.py -opt options/df2k/train_bicubic_noise.yml
    • checkpoint dir is in '../experiments'

Track 2

  1. prepare training data

    • Use KernelGAN to generate kernels from source images. Clone the repo here. Replace SOURCE_PATH with specific path and run :

      cd KernelGAN
CUDA_VISIBLE_DEVICES=4,5,6,7 python3 train.py --X4 --input-dir SOURCE_PATH

    • specify dataset paths in './preprocess/path.yml' and generated KERNEL_PATH to kernel create kernel dataset: python3 ./preprocess/create_kernel_dataset.py --dataset dped --artifacts clean --kernel_path KERNEL_PATH

    • run the below command to collect high frequency noise from Source: python3 ./preprocess/collect_noise.py --dataset dped --artifacts clean

  2. train SR model

    • Modify the configuration file options/dped/train_kernel_noise.yml
    • run command : CUDA_VISIBLE_DEVICES=4,5,6,7 python3 train.py -opt options/dped/train_kernel_noise.yml
    • checkpoint dir is in '../experiments'