GRL for Image Restoration

This repository is an official implementation of the paper Efficient and Explicit Modelling of Image Hierarchies for Image Restoration.

By Yawei Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx, Rakesh Ranjan, Radu Timofte, and Luc Van Gool

News

• Mar 20, 2023: 🚀 GRL is released！
  • [ ] GRL-B/S/T model for image denosing.
  • [ ] GRL-B/S/T model for image denosing.
  • [ ] GRL-B/S/T model for single-image super-resolution.
  • [ ] GRL-B model for single-image motion deblurring.
  • [ ] GRL-B model for image defocus deblurring.
  • [ ] GRL-B model for real-world image super-resolution.
  • [ ] GRL-B model for image demosaicking.
  • [ ] GRL-S model for JPEG compression artifacts removal.
• Feb 28, 2023: 🚀 GRL is accepted to CVPR 2023!

Coming soom

• [ ] LightningIR: A general framework for image restoration.
• [ ] LSDIR: A large-scale dataset for image restoration.

Introduction

GRL provides a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration. To achieve that, we start by analyzing two important properties of natural images including cross-scale similarity and anisotropic image features. Inspired by that, we propose the anchored stripe self-attention which achieves a good balance between the space and time complexity of self-attention and the modelling capacity beyond the regional range. Then we propose a new network architec- ture dubbed GRL to explicitly model image hierarchies in the Global, Regional, and Local range via anchored stripe self-attention, window self-attention, and channel attention enhanced convolution. Finally, the proposed network is applied to 7 image restoration types, covering both real and synthetic settings. The proposed method sets the new state-of-the-art for several of those.

How to Use the Code?

1. conda create -n LightningIR python=3.8
2. conda activate LightningIR
3. pip install -r requirements.txt
4. prepare the dataset
5. download the pretrained models

6. torchx run -- -j 1x2 -- \
       -m training=False gpus=2 experiment=dm/grl model=grl/grl_small \
       load_state_dict=True pretrained_checkpoint="${MODEL_ZOO}/GRL/dm_grl_small.ckpt"

Main Results

Results

Image denoising (click to expand)

Image SR (click to expand)

Single-Image Motion Deblur (click to expand)

Defocus Deblur (click to expand)

JPEG Compression Artifact Removal (click to expand)

Image Demosaicking (click to expand)

Real-World Image SR (click to expand)

Citation

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{li2023grl,
  title={Efficient and Explicit Modelling of Image Hierarchies for Image Restoration},
  author={Yawei Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx, Rakesh Ranjan, Radu Timofte, and Luc Van Gool},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2023}
}