By Di Wang, Jinyuan Liu, Xin Fan, and Risheng Liu
[2022-07-14] The pretrained models of registration network (MRRN) and fusion network (DIFN) are available!
[2022-06-21] The CPSTN is available!
[2022-05-30] The Chinese translation of our paper is available, please enjoy it! [中译版本]
[2022-05-25] Our paper is available online! [arXiv version]
cd ./data
python get_test_data.py
In 'Trainer/train_reg.py', deformable infrared images are generated in real time by default during training.
cd ./data
python get_svs_map.py
You can use the pseudo infrared images [link code: qqyj] generated by our CPSTN to train/test the registration process:
cd ./Trainer
python train_reg.py
cd ./Test
python test_reg.py
Please download the [pretrained model](https://pan.baidu.com/s/199dqOLHyJS9aY5YecuVglA) (code: hk25) of the registration network MRRN.
If you want to generate pseudo-infrared images using our CPSTN for other datasets, you can directly run following commands:
## testing
cd ./CPSTN
python test.py --dataroot datasets/rgb2ir/RoadScene/testA --name rgb2ir_paired_Road_edge_pretrained --model test --no_dropout --preprocess none
## training
cd ./CPSTN
python train.py --dataroot ./datasets/rgb2ir/RoadScene --name rgb2ir_paired_Road_edge --model cycle_gan --dataset_mode unaligned
The training and testing data of our CPSTN can be downloaded from: [datasets](https://pan.baidu.com/s/1-U1n945ykHFU7yrEHwGC9Q) (code: u386)
Please download the [pretrained model](https://pan.baidu.com/s/1JO4hjdaXPUScCI6oFtPEnQ) (code: i9ju) of CPSTN and put it into folder './CPSTN/checkpoints/pretrained/'
If you tend to train Registration and Fusion processes separately, You can run following commands:
cd ./Trainer
python train_reg.py
cd ./Trainer
python train_fuse.py
The corresponding test code 'test_reg.py' and 'test_fuse.py' can be found in 'Test' folder. Please download the [pretrained model](https://pan.baidu.com/s/1GZrYrg_qzAfQtoCrZLJsSw) (code: 0rbm) of fusion network DIFN.
If you tend to train Registration and Fusion processes jointly, You can run following command:
cd ./Trainer
python train_reg_fusion.py
The corresponding test code 'test_reg_fusion.py' can be found in 'Test' folder.
Please download the following datasets:
Please download the pseudo infrared images generated by our CPSTN:
Please download the registered infrared images by our UMF:
Please download the fused images by our UMF:
@inproceedings{UMF,
author = {Di Wang and
Jinyuan Liu and
Xin Fan and
Risheng Liu},
title = {Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration},
booktitle = {IJCAI},
pages = {3508--3515},
year = {2022}
}