Authors: Tao Zhou, Huazhu Fu, Geng Chen, Yi Zhou, Deng-Ping Fan, and Ling Shao.
RGB-D saliency detection has attracted increasing attention, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing works often focus on learning a shared representation through various fusion strategies, with few methods explicitly considering how to preserve modality-specific characteristics. In this paper, taking a new perspective, we propose a specificitypreserving network (SP-Net) for RGB-D saliency detection, which benefits saliency detection performance by exploring both the shared information and modality-specific properties (e.g., specificity). Specifically, two modality-specific networks and a shared learning network are adopted to generate individual and shared saliency maps. A crossenhanced integration module (CIM) is proposed to fuse cross-modal features in the shared learning network, which are then propagated to the next layer for integrating cross-level information. Besides, we propose a multi-modal feature aggregation (MFA) module to integrate the modality-specific features from each individual decoder into the shared decoder, which can provide rich complementary multi-modal information to boost the saliency detection performance. Further, a skip connection is used to combine hierarchical features between the encoder and decoder layers. Experiments on six benchmark datasets demonstrate that our SP-Net outperforms other state-of-the-art methods.
Figure 1: The overall architecture of the proposed SP-Net.
Figure 2: Visual comparisons of our method and eight state-of-the-art methods.
The training and testing experiments are conducted using PyTorch with one NVIDIA Tesla V100 GPU with 32 GB memory.
Configuring your environment (Prerequisites):
pip install -r requirements.txt
.Downloading necessary data:
Downloading training dataset (download link (Google Drive)) and move it into ./Data/
.
Downloading testing dataset (download link (Google Drive)) and move it into ./Data/
.
Downloading pretrained weights (download link (Google Drive)) and move it into ./Checkpoint/SPNet/
.
Train Configuration:
train.py
to train our model.Test Configuration:
After you download all the pre-trained model and testing dataset, just run test_produce_maps.py
to generate the final prediction map, then run test_evaluation_maps.py
to obtain the final quantitative results.
You can also download predicted saliency maps (download link (Google Drive)) and move it into ./Predict_maps/
, then then run test_evaluation_maps.py
.
Our evaluation is implemented by python, please refer to test_evaluation_maps.py
Please cite our paper if you find the work useful, thanks!
@inproceedings{zhouiccv2021,
title={Specificity-preserving RGB-D Saliency Detection},
author={Zhou, Tao and Fu, Huazhu and Chen, Geng and Zhou, Yi and Fan, Deng-Ping and Shao, Ling},
booktitle={International Conference on Computer Vision (ICCV)},
year={2021},
}
@inproceedings{zhoucvmj2022,
title={Specificity-preserving RGB-D Saliency Detection},
author={Zhou, Tao and Fan, Deng-Ping and Chen, Geng and Zhou, Yi and Fu, Huazhu},
booktitle={Computational Visual Media},
year={2022},
}