SCTransNet: Spatial-channel Cross Transformer Network for Infrared Small Target Detection [Paper] [Weight]

Shuai Yuan, Hanlin Qin, Xiang Yan, Naveed Akhtar, Aimal Main, IEEE Transactions on Geoscience and Remote Sensing 2024.

SCTransNet 是PRCV 2024、ICPR 2024 Track 1、ICPR 2024 Track 2 三项比赛冠军方案的 Baseline, 同时也是多个优胜算法的Baselines. [Paper]

Bilibili 视频分享

https://www.bilibili.com/video/BV1kr421M7wx/

极市平台 推文分享

https://mp.weixin.qq.com/s/H7KLmtFX7j09f-Xc6X1FRw

If the implementation of this repo is helpful to you, just star it！⭐⭐⭐

Chanlleges and inspiration

Structure

Introduction

We present a Spatial-channel Cross Transformer Network (SCTransNet) to the IRSTD task. Experiments on both public (e.g., SIRST, NUDT-SIRST, IRSTD-1K) demonstrate the effectiveness of our method. Our main contributions are as follows:

1. We propose SCTransNet, leveraging spatial-channel cross transformer blocks (SCTB) to predict the context of targets and backgrounds in the deeper network layers.

2. A spatial-embedded single-head channel-cross attention (SSCA) module is utilized to foster semantic interactions across all feature levels and learn the long-range context.

3. We devise a novel complementary feed-forward network (CFN) by crossing spatial-channel information to enhance the semantic difference between the target and background.

Usage

1. Data

• Our project has the following structure:

  ├──./datasets/
  │    ├── IRSTD-1K
  │    │    ├── images
  │    │    │    ├── XDU0.png
  │    │    │    ├── XDU1.png
  │    │    │    ├── ...
  │    │    ├── masks
  │    │    │    ├── XDU0.png
  │    │    │    ├── XDU1.png
  │    │    │    ├── ...
  │    │    ├── img_idx
  │    │    │    ├── train_IRSTD-1K.txt
  │    │    │    ├── test_IRSTD-1K.txt
  │    ├── NUDT-SIRST
  │    │    ├── images
  │    │    │    ├── 000001.png
  │    │    │    ├── 000002.png
  │    │    │    ├── ...
  │    │    ├── masks
  │    │    │    ├── 000001.png
  │    │    │    ├── 000002.png
  │    │    │    ├── ...
  │    │    ├── img_idx
  │    │    │    ├── train_NUDT-SIRST.txt
  │    │    │    ├── test_NUDT-SIRST.txt
  │    ├── ...
  │    ├── ...
  │    ├── SIRST3
  │    │    ├── images
  │    │    │    ├── XDU0.png
  │    │    │    ├── XDU1.png
  │    │    │    ├── ...
  │    │    ├── masks
  │    │    │    ├── XDU0.png
  │    │    │    ├── XDU1.png
  │    │    │    ├── ...
  │    │    ├── img_idx
  │    │    │    ├── train_SIRST3.txt
  │    │    │    ├── test_SIRST3.txt
  

2. Train.

python train.py

3. Test and demo.

python test.py

Results and Trained Models

Qualitative Results

Quantitative Results on Mixed SIRST, NUDT-SIRST, and IRSTD-1K

*This code is highly borrowed from IRSTD-Toolbox. Thanks to Xinyi Ying.

*This code is highly borrowed from UCTransNet. Thanks to Haonan Wang.

*The overall repository style is highly borrowed from DNA-Net. Thanks to Boyang Li.

Citation

If you find the code useful, please consider citing our paper using the following BibTeX entry.

@ARTICLE{10486932,
  author={Yuan, Shuai and Qin, Hanlin and Yan, Xiang and Akhtar, Naveed and Mian, Ajmal},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={SCTransNet: Spatial-Channel Cross Transformer Network for Infrared Small Target Detection}, 
  year={2024},
  volume={62},
  number={},
  pages={1-15},
  keywords={Semantics;Transformers;Decoding;Feature extraction;Task analysis;Object detection;Visualization;Convolutional neural network (CNN);cross-attention;deep learning;infrared small target detection (IRSTD);transformer},
  doi={10.1109/TGRS.2024.3383649}}

Contact

Welcome to raise issues or email to yuansy@stu.xidian.edu.cn or yuansy2@student.unimelb.edu.au for any question regarding our SCTransNet.