📖 SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution

[Paper] [Supp]

Mingjun Zheng, Long Sun, Jiangxin Dong, and Jinshan Pan
IMAG Lab, Nanjing University of Science and Technology

Network architecture of the proposed SMFANet. The proposed SMFANet consists of a shallow feature extraction module, feature modulation blocks, and a lightweight image reconstruction module. Feature modulation block contains one self-modulation feature aggregation (SMFA) module and one partial convolution-based feed-forward network (PCFN).

News

[2024-09-26] The paper is available Here.
[2024-08-04] We add the scripts for feature visualization and chart plotting.
[2024-07-16] We add 🤗Hugging Face Demo.
[2024-07-01] Our SMFANet is accepted by ECCV 2024.
[2024-06-25] Our SMFANet places 2nd and 3rd in the Parameters and FLOPs sub-track of the NTIRE2024 ESR.

Requirements
- Python 3.8, PyTorch >= 1.8
- BasicSR 1.4.2
- Platforms: Ubuntu 18.04, cuda-11

Installation

# Clone the repo
git clone https://github.com/Zheng-MJ/SMFANet.git
# Install dependent packages
cd SMFANet
conda create --name smfan python=3.8
conda activate smfan
pip install -r requirements.txt
# Install BasicSR
python setup.py develop

You can also refer to this INSTALL.md for installation

Data Preparation

Please refer to datasets/REDAME.md for data preparation.

Training

Run the following commands for training:

# train SMFANet for x4 effieicnt SR
python basicsr/train.py -opt options/train/SMFANet/SMFANet_DIV2K_100w_x4SR.yml
# train SMFANet+ for x4 effieicnt SR
python basicsr/train.py -opt options/train/SMFANet/SMFANet_plus_DIV2K_100w_x4SR.yml

Testing

Download the testing dataset.

Run the following commands:

# test SMFANet for x4 efficient SR
python basicsr/test.py -opt options/test/SMFANet_DF2K_x4SR.yml

The test results will be in './results'.

Pretrained Model \& Visual Results

Google Drive | Huggingface

TensorRT Optimization

The script for exporting TensorRT model is available at to_tensorrt/READEME.md

Hugging Face Demo

The Hugging Face Demo is available here.

Plotting Script

The script for feature visualization and chart plotting is available at plt/README.md.

Experimental Results

Comparison with CNN-based lightweight SR methods
Comparison with ViT-based lightweight SR methods
Memory and running time comparisons on x4 SR
Visual comparisons for x4 SR on the Urban100 dataset
Comparison of local attribution maps (LAMs) and diffusion indices (DIs)
The power spectral density (PSD) visualizations of feature

Citation

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{smfanet,
    title={SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution},
    author={Zheng, Mingjun and Sun, Long and Dong, Jiangxin and Pan, Jinshan},
    booktitle={ECCV},
    year={2024}
 }

Acknowledgement

This code is based on BasicSR toolbox. Thanks for the awesome work.

Contact

If you have any questions, please feel free to reach me out at mingjunzheng@njust.edu.cn

Zheng-MJ / SMFANet

readme