search

YanchaoYang / FDA

Fourier Domain Adaptation for Semantic Segmentation
501 stars 81 forks source link

readme

FDA: Fourier Domain Adaptation for Semantic Segmentation.

This is the Pytorch implementation of our FDA paper published in CVPR 2020.

Domain adaptation via style transfer made easy using Fourier Transform. FDA needs no deep networks for style transfer, and involves no adversarial training. Below is the diagram of the proposed Fourier Domain Adaptation method:

Step 1: Apply FFT to source and target images.

Step 2: Replace the low frequency part of the source amplitude with that from the target.

Step 3: Apply inverse FFT to the modified source spectrum.

Image of FDA

Usage

  1. FDA Demo

    python3 FDA_demo.py

    An example of FDA for domain adaptation. (source: GTA5, target: CityScapes, with beta 0.01)

    Image of Source

  2. Sim2Real Adaptation Using FDA (single beta)

    python3 train.py --snapshot-dir='../checkpoints/FDA' --init-weights='../checkpoints/FDA/init_weight/DeepLab_init.pth' --LB=0.01 --entW=0.005 --ita=2.0 --switch2entropy=0

    Important: use the original images for FDA, then do mean subtraction, normalization, etc. Otherwise, will be numerical artifacts.

    DeepLab initialization can be downloaded through this link.

    LB: beta in the paper, controls the size of the low frequency window to be replaced.

    entW: weight on the entropy term.

    ita: coefficient for the robust norm on entropy.

    switch2entropy: entropy minimization kicks in after this many steps.

  3. Evaluation of the Segmentation Networks Adapted with Multi-band Transfer (multiple betas)

    python3 evaluation_multi.py --model='DeepLab' --save='../results' --restore-opt1="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_01" --restore-opt2="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_05" --restore-opt3="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_09"

    Pretrained models on the GTA5 -> CityScapes task using DeepLab backbone can be downloaded here.

    The above command should output: ===> mIoU19: 50.45 ===> mIoU16: 54.23 ===> mIoU13: 59.78

  4. Get Pseudo Labels for Self-supervised Training

    python3 getSudoLabel_multi.py --model='DeepLab' --data-list-target='./dataset/cityscapes_list/train.txt' --set='train' --restore-opt1="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_01" --restore-opt2="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_05" --restore-opt3="../checkpoints/FDA/gta2city_deeplab/gta2city_LB_0_09"

  5. Self-supervised Training with Pseudo Labels

    python3 SStrain.py --model='DeepLab' --snapshot-dir='../checkpoints/FDA' --init-weights='../checkpoints/FDA/init_weight/DeepLab_init.pth' --label-folder='cs_pseudo_label' --LB=0.01 --entW=0.005 --ita=2.0

  6. Other Models

    VGG initializations can be downloaded through this link.

    Pretrained models on the Synthia -> CityScapes task using DeepLab backbone link.

    Pretrained models on the GTA5 -> CityScapes task using VGG backbone link.

    Pretrained models on the Synthia -> CityScapes task using VGG backbone link.

Acknowledgment

Code adapted from BDL.