High-Precision Dichotomous Image Segmentation with Frequency and Scale Awareness (TNNLs-2024)

Authors: Qiuping Jiang, Jinguang Cheng^*, Zongwei Wu, Runmin Cong, Radu Timofte.

1. Preface

• This repository provides code for "High-Precision Dichotomous Image Segmentation with Frequency and Scale Awareness" TNNLs-2024.

2. Overview

2.1. Introduction

Dichotomous Image Segmentation (DIS) with rich fine-grained details within a single image is a challenging task. Despite the plausible results achieved by deep learning-based methods, most of them fail to segment generic objects when the boundary is cluttered with the background. In fact, the gradual decrease in feature map resolution during the encoding stage and the misleading texture clue may be the main issues. To handle these issues, we devise a novel frequency- and scale-aware deep neural network (FSANet) for high-precision DIS. The core of our proposed FSANet is twofold. First, a multi-modality fusion (MF) module that integrates the information in spatial and frequency domains is adopted to enhance the representation capability of image features. Second, a collaborative scale fusion module (CSFM) which deviates from the traditional serial structures is introduced to maintain high resolution during the entire feature encoding stage. In the decoder side, we introduce hierarchical context fusion (HCF) and selective feature fusion (SFF) modules to infer the segmentation results from the output features of the CSFM module. We conduct extensive experiments on several benchmark datasets and compare our proposed method with existing SOTA methods. The experimental results demonstrate that our FSANet achieves superior performance both qualitatively and quantitatively.

2.2. Framework Overview

Figure 1: Architecture Overview

2.3. Qualitative Results

Figure 2: Qualitative Results.

3. Proposed Baseline

3.1. Training/Testing

The training and testing experiments are conducted using PyTorch with double 3090 GPU of 24 GB Memory.

1. Configuring your environment (Prerequisites):

   • Creating a virtual environment in terminal: conda create -n FSANet python=3.8.

   • Installing necessary packages: pip install -r requirements.txt. (PS: You can also select some of these packages to install that you need.)

2. Downloading necessary data:

   • downloading training/testing dataset and move it into data, which can be found in this (Google Drive).

   • downloading Res2Net weights and move it into ./models/res2net50_v1b_26w_4s-3cf99910.pth(Google Drive) or (BaiduNetdisk)

   • downloading pvt-v2 weights and move it into ./FSANet/pvt_v2_b2.pth(Google Drive) or (BaiduNetdisk)

3. Training Configuration:

   • Assigning your costumed path, like --train_save and --train_path in train.py.

4. Testing Configuration:

   • After you download all the pre-trained model and testing dataset, just run test.py to generate the final prediction map: replace your trained model directory (--pth_path).

5. Comparing with your model:

   • We provide visualizations images that you can download form (Google Drive) or (Baidudisk)

3.2 Evaluating your trained model:

One-key evaluation is written in python code (revised from link)

4. Citation

⬆ back to top