search

isht7 / pytorch-deeplab-resnet

DeepLab resnet v2 model in pytorch
MIT License
602 stars 117 forks source link

Bad mIOU tested with provided model #41

Open tengteng95 opened 6 years ago

tengteng95 commented 6 years ago

Hello! I follow the Readme to test the mIOU with given pretrained caffe model on VOC. However, I get unexpected Mean iou = 0.046470389101241724. Do you have any suggestions about this problem? By the way, why the image is only substracted by the mean value of each channel without normlization to range such as [0,1] or [-1,1] in the code?

tengteng95 commented 6 years ago

I use the model downloaded from here

suhangpro commented 6 years ago

I am facing the same issue. The number I got is 0.04647076057203757. Using the provided .pth file without any code modification.

tengteng95 commented 6 years ago

@suhangpro hi, the problem is that opencv cannot read image in mode 'p' correctly. Reading gt image with PIL.Image will solve the problem.

suhangpro commented 6 years ago

Thanks, @h982639009 ! After that change, I got: pytorch 20 Mean iou = 0.7645974317511359. This is even higher than the number 76.35 reported in the paper (Table. 4 row 6). Someone reported 76.42 in #5 . I don't see any randomness in the forward network inference, so the slight difference is still strange.

chenyzh28 commented 6 years ago

@suhangpro, hi, may I ask how do you get the mean iou of 0.7645? Through evalpyt.py or evalpyt2.py? Becausce I got 0.7802 through evalpyt2.py and 0.7180 through evalpyt.py.

suhangpro commented 6 years ago

I did the fix @h982639009 mentioned, and made no other changes.

chenyzh28 commented 6 years ago

@suhangpro , do you know how to convert the gt and the output images to color images?

chenyzh28 commented 6 years ago

@h982639009 , may I ask what mode 'p' means? How can i can convert images of mode 'p' to RGB images？