Multi-Anchor Active Domain Adaptation for Semantic Segmentation

Munan Ning, Donghuan Lu, Dong Wei†, Cheng Bian, Chenglang Yuan, Shuang Yu, Kai Ma, Yefeng Zheng

[Paper] [PPT] [Graphic Abstract]

Table of Contents

• Introduction
• Requirements
• Usage
• License
• Notes

Introduction

This respository contains the MADA method as described in the ICCV 2021 Oral paper "Multi-Anchor Active Domain Adaptation for Semantic Segmentation".

Requirements

The code requires Pytorch >= 0.4.1 with python 3.6. The code is trained using a NVIDIA Tesla V100 with 32 GB memory. You can simply reduce the batch size in stage 2 to run on a smaller memory.

Usage

1. Preparation:

   • Download the GTA5 dataset as the source domain, and the Cityscapes dataset as the target domain.
   • Download the weights and features. Move features to the MADA directory.

2. Setup the config files.

   • Set the data paths
   • Set the pretrained model paths

3. Training-quick

   • To run the code with our weights and anchors (anchors/cluster_centroids_full_10.pkl):

     python3 train_active_stage1.py
     python3 train_active_stage2.py

   • During the training, the generated files (log file) will be written in the folder 'runs/..'.

4. Evaluation

   • Set the config file for test (configs/test_from_city_to_gta.yml):
   • Run:

     python3 test.py

     to see the results.

5. Training-whole process

   • Setting the config files.
   • Stage 1:
   • 1-save_feat_source.py: get the './features/full_dataset_objective_vectors.pkl'

     python3 save_feat_source.py

   • 2-cluster_anchors_source.py: cluster the './features/full_dataset_objective_vectors.pkl' to './anchors/cluster_centroids_full_10.pkl'

     python3 cluster_anchors_source.py

   • 3-select_active_samples.py: select active samples with './anchors/cluster_centroids_full_10.pkl' to 'stage1_cac_list_0.05.txt'

     python3 select_active_samples.py

   • 4-train_active_stage1.py: train stage1 model with anchors './anchors/cluster_centroids_full_10.pkl' and active samples 'stage1_cac_list_0.05.txt', get the 'from_gta5_to_cityscapes_on_deeplab101_best_model_stage1.pkl', which is stored in the runs/active_from_gta_to_city_stage1

     python3 train_active_stage1.py

• Stage 2:
• 1-save_feat_target.py: get the './features/target_full_dataset_objective_vectors.pkl.pkl'

  python3 save_feat_target.py

• 2-cluster_anchors_target.py: cluster the './features/target_full_dataset_objective_vectors.pkl' to './anchors/cluster_centroids_full_target_10.pkl'

  python3 cluster_anchors_target.py

• 3-train_active_stage2.py: train stage2 model with anchors './anchors/cluster_centroids_full_target_10.pkl' and active samples 'stage1_cac_list_0.05.txt', get the 'from_gta5_to_cityscapes_on_deeplab101_best_model_stage2.pkl'

  python3 train_active_stage2.py

License

MIT

The code is heavily borrowed from the CAG_UDA (https://github.com/RogerZhangzz/CAG_UDA).

If you use this code and find it usefule, please cite:

@inproceedings{ning2021multi,
  title={Multi-Anchor Active Domain Adaptation for Semantic Segmentation},
  author={Ning, Munan and Lu, Donghuan and Wei, Dong and Bian, Cheng and Yuan, Chenglang and Yu, Shuang and Ma, Kai and Zheng, Yefeng},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={9112--9122},
  year={2021}
}

Notes

The anchors are calcuated based on features captured by decoders.

In this paper, we utilize the more powerful decoder in DeeplabV3+, it may cause somewhere unfair. So we strongly recommend the ProDA which utilize origin DeeplabV2 decoder.