The official repo of the CVPR 2024 paper (Highlight), Multi-view Aggregation Network for Dichotomous Image Segmentation
Dichotomous Image Segmentation (DIS) has recently emerged towards high-precision object segmentation from high-resolution natural images. When designing an effective DIS model, the main challenge is how to balance the semantic dispersion of high-resolution targets in the small receptive field and the loss of high-precision details in the large receptive field. Existing methods rely on tedious multiple encoder-decoder streams and stages to gradually complete the global localization and local refinement.
Human visual system captures regions of interest by observing them from multiple views. Inspired by it, we model DIS as a multi-view object perception problem and provide a parsimonious multi-view aggregation network (MVANet), which unifies the feature fusion of the distant view and close-up view into a single stream with one encoder-decoder structure. Specifically, we split the high-resolution input images from the original view into the distant view images with global information and close-up view images with local details. Thus, they can constitute a set of complementary multi-view low-resolution input patches.
Moreover, two efficient transformer-based multi-view complementary localization and refinement modules (MCLM & MCRM) are proposed to jointly capturing the localization and restoring the boundary details of the targets.
We achieves state-of-the-art performance in terms of almost all metrics on the DIS benchmark dataset.
We have optimized the code and achieved an enhanced FPS performance, reaching 15.2.
Here are some of our visual results:
python train.py
./utils/config.py
(line 4~8)Replace the existing path with the path to your saved model in ./predict.py
(line 14)
You can also download our trained model at Google Drive.
python predict.py
./test.py
(line 17) and start testing:
python test.py
You can get our prediction maps at Google Drive.
@article{yu2024multi,
title={Multi-view Aggregation Network for Dichotomous Image Segmentation},
author={Yu, Qian and Zhao, Xiaoqi and Pang, Youwei and Zhang, Lihe and Lu, Huchuan},
journal={arXiv preprint arXiv:2404.07445},
year={2024}
}