search

lhoyer / DAFormer

[CVPR22] Official Implementation of DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation
Other
466 stars 92 forks source link

Performance gap on Synthia to Cityscapes #18

Closed Pranav-Goyal closed 2 years ago

Pranav-Goyal commented 2 years ago

Hi,

I tried reproducing results for adaptation from Synthia to Cityscapes by running python run_experiments.py --exp 7. The obtained results were ~51 mIOU (87 aAcc, 62 mAcc) which is short of the reported 60.9 mIOU. I used a single GeForce RTX 2080 Ti GPU to run the experiment.

I am trying to match the reported results and seek your help on this.

Thank you!

lhoyer commented 2 years ago

The evaluation script calculates the mIoU for all 19 Cityscapes classes. However, Synthia contains only labels for 16 of these classes. Therefore, it is a common practice in UDA to report the mIoU for Synthia->Cityscapes only on these 16 classes. As the Iou for the 3 missing classes is 0, you can do the conversion mIoU16 = mIoU19 * 19 / 16

Pranav-Goyal commented 2 years ago

Understood. The scores match after the conversion. Thank you for clarifying!