search

wasidennis / AdaptSegNet

Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR 2018 (spotlight)
847 stars 203 forks source link

Evaluation of the model #11

Closed FrancisLeon closed 6 years ago

FrancisLeon commented 6 years ago

Hello,

I have evaluate the model GTA2Cityscapes_multi-ed35151c.pth provided in your link. But, I find the result of it is a bit different from your paper. So I wonder it's just part of the checkpoints. Thanks.

items GTA2Cityscapes_multi-ed35151c.pth Paper
road: 85.11 86.5
sidewalk: 36.14 36.0
building: 79.0 79..9
wall: 28.6 23.4
fence: 22.03 23.3
pole: 18.89 23.9
light: 31.74 35.2
sign: 12.54 14.8
vegetation: 81.37 83.4
terrain: 36.4 33.3
sky: 72.56 75.6
person: 52.15 58.5
rider: 24.81 27.6
car: 69.34 73.7
truck: 33.84 32.5
bus: 34.99 35.4
train: 3.27 3.9
motocycle: 26.02 30.1
bicycle: 26.43 28.1
mIoU: 40.8 42.4
hfslyc commented 6 years ago

Hi,

Thanks for pointing it out. I suspect that's because we fixed a bug for the deeplab model in #4. And the pretrained model was trained on the original deeplab model.

We'll verify this issue and go back to you soon.

hfslyc commented 6 years ago

@FrancisLeon I assume you are using PyTorch0.4? We just found that if you use pytorch0.4 you will need to change L108 to:

interp = nn.Upsample(size=(1024, 2048), mode='bilinear', align_corners=True)

This will lead to the same evaluation results as in the paper. We'll update the code to be compatible with Pytorch0.4 soon

hfslyc commented 6 years ago

the temporary fix is in the commit 5c5e03e37ac

FrancisLeon commented 6 years ago

Thanks for your reply.