This is official Pytorch implementation of "Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network"
The overall framework of the proposed semantic-aware infrared and visible image fusion algorithm.
The architecture of the real-time infrared and visible image fusion network based on gradient residual dense block.
Run **CUDA_VISIBLE_DEVICES=0 python train.py**
to train your model.
The training data are selected from the MFNet dataset. For convenient training, users can download the training dataset from here, in which the extraction code is: bvfl.
The MFNet dataset can be downloaded via the following link: https://drive.google.com/drive/folders/18BQFWRfhXzSuMloUmtiBRFrr6NSrf8Fw.
The MFNet project address is: https://www.mi.t.u-tokyo.ac.jp/static/projects/mil_multispectral/.
Run **CUDA_VISIBLE_DEVICES=0 python test.py**
to test the model.
For quantitative assessments, please follow the instruction to modify and run . /Evaluation/test_evaluation.m .
Qualitative comparison of SeAFusion with 9 state-of-the-art methods on 00633D image from the MFNet dataset.
Segmentation results for infrared, visible and fused images from the MFNet dataset. The segmentation models are re-trained on infrared, visible and fused image sets.
Each two rows represent a scene.
Segmentation results for infrared, visible and fused images from the MFNet dataset. The segmentation model is Deeplabv3+, pre-trained on the Cityscapes dataset. Each
two rows represent a scene.
Object detection results for infrared, visible and fused images from the MFNet dataset. The YOLOv5 detector, pre-trained on the Coco dataset is deployed to achieve
object detection.
@article{TANG202228SeAFusion,
title = {Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network},
journal = {Information Fusion},
volume = {82},
pages = {28-42},
year = {2022},
issn = {1566-2535}
}