search

scaelles / DEXTR-PyTorch

Deep Extreme Cut http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr
GNU General Public License v3.0
843 stars 153 forks source link

Cannot reproduce the same result as provided Dextr_pascal-sbd model #14

Closed Valentine233 closed 5 years ago

Valentine233 commented 5 years ago

Thanks for your great work!

When I reproduce your results of Dextr_pascal-sbd, I cannot get the same performance (IoU=91.5).

Firstly, I just evaluate with your provided model "dextr_pascal-sbd.pth" and I can get the IoU of 91.5. However, when I try to train the model by myself, I only get the IoU of 86.2, with a gap of 5.3. I have barely changed your codes.

Here is my setup:

Could you please give me some suggestions :)

kmaninis commented 5 years ago

Hi, could you try with different thresholds during evaluation? We have checked reproducibility at least 3 times, for different versions of PyTorch. Also, default behaviour of upsampling has changed (aligned corners), which should be fine if you stick to the given code.

Valentine233 commented 5 years ago

Thanks for your suggestion.

Changing thresholds is helpful!

kmaninis commented 5 years ago

Hi, let us know whether this was the only problem.