PPI Edge Infused Spatial–Spectral Adaptive Residual Network

This repo is the official implementation of the following paper:

"PPI Edge Infused Spatial–Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing" (TGRS 2023) [Paper].

Introduction

We present a pseudo-panchromatic image (PPI) edge-infused spatial–spectral adaptive residual network (PPIE-SSARN) for MSFA image demosaicing.

The proposed two-branch model deploys a residual subbranch to adaptively compensate for the spatial and spectral differences of reconstructed multispectral images and a PPI edge infusion subbranch to enrich the edge-related information. Moreover, we design an effective mosaic initial feature extraction module with a spatial- and spectral-adaptive weight-sharing strategy whose kernel weights can change adaptively with spatial locations and spectral bands to avoid artifacts and aliasing problems.

Datasets

• ARAD-1K Dataset

  • Training [Baidu] (Code: arad)
  • Testing [Baidu] (Code: arad) / [Google]

• Chikusei Dataset

  • Training [Baidu] (Code: chik) / [Google]
  • Testing [Baidu] (Code: chik) / [Google]

Install

• Clone this repo:

  git clone https://github.com/bowenzhao-zju/PPIE-SSARN
  cd PPIE-SSARN

• Build the DDMF module:

  cd model/ddf
  python setup.py install
  mv build/lib*/* .

Train

• Set the data path and the hyperparameters for training in config.py.

• Run train.py:

  python train.py

Test

Quick Start

• Download the weights file valid_best.pth and place it in checkpoint.

• Run test.py:

  python test.py

• The results are in checkpoint/pred.

Test

• Set the data path and the weight file (*.pth) path for testing in config.py.

• Run test.py:

  python test.py

Results

Visual comparisons for demosaicing results on the ARAD-1K dataset. Our method is compared with five alternatives, that is, MCAN [Code], DPDNet [Code], 3D-ResNet, PPID [Code], and WB. The first column shows the GT. In addition, zoomed-in views of selected regions are provided. Please zoom in to see the details.

Citation

If you find the code and datasets helpful in your research work, please cite the following paper:

@ARTICLE{10188849,
  author={Zhao, Bowen and Zheng, Jiesi and Dong, Yafei and Shen, Ning and Yang, Jiangxin and Cao, Yanlong and Cao, Yanpeng},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={PPI Edge Infused Spatial–Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing}, 
  year={2023},
  volume={61},
  number={},
  pages={1-14},
  doi={10.1109/TGRS.2023.3297250}}