search

layumi / Seg-Uncertainty

IJCAI2020 & IJCV2021 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo
https://arxiv.org/abs/1912.11164
MIT License
387 stars 51 forks source link

can not get good performance #4

Closed panzhang0212 closed 4 years ago

panzhang0212 commented 4 years ago

Hi, I have some problems 1.for your release model: stage 1 model (SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5), I test this model using your code, the mIoU is only 38.07, much lower than 45.5(stage 1 miou in MRNet paper). stage2 model(1280x640_restore_ft_GN_batchsize9_512x256_pp_ms_me0_classbalance7_kl0_lr1_drop0.2_seg0.5_BN_80_255_0.8_Noaug) is 50.34 , which is same as the report result in second paper.

  1. train result I use your released stage 1 model to generate pseudo label, then train stage 2, However, the performance is low: 43.89 for 25000.pth, 42.69 for 50000.pth, 41.98 for 100000.pth, much lower than the result in your paper. And I do not change any code except variable DATA_DIRECTORY
layumi commented 4 years ago

image image

As shown in the figure, I just re-run the test code. 

python evaluate_cityscapes.py --restore-from ./snapshots/SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5/GTA5_25000.pth --batchsize 2

And my test result is 45.5

layumi commented 4 years ago

image Could you check the md5sum first? Thanks a lot.

panzhang0212 commented 4 years ago

snipaste_20200530_141453 your release de model is GTA50000.pth

panzhang0212 commented 4 years ago

Could you clone the code and released SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5/GTA5_50000.pth, then try the test code again? thanks

layumi commented 4 years ago

It is my fault. I have update the download link https://drive.google.com/file/d/1smh1sbOutJwhrfK8dk-tNvonc0HLaSsw/view?usp=sharing

panzhang0212 commented 4 years ago

Thanks your help!