search

scaelles / DEXTR-PyTorch

Deep Extreme Cut http://www.vision.ee.ethz.ch/~cvlsegmentation/dextr
GNU General Public License v3.0
843 stars 153 forks source link

Results of dextr_pascal and dextr_coco models #21

Closed ahmedhshahin closed 4 years ago

ahmedhshahin commented 5 years ago

Hi,

When I tried to reproduce the results using the provided pretrained model, I got some different results. For dextr_pascal, you report an accuracy of 90.5. However, I got an accuracy of 90.1 at threshold 0.5. For dextr_coco, you report an accuracy of 87.9. I got an accuracy of 90.9 at threshold 0.8. I am just feeding forward the validation data to your pretrained model, so I think I should get the same results. Did you encounter any similar behaviours or have any justifications for this significant difference (in case of dextr_coco)?

Thank you

scaelles commented 5 years ago

Hello, Have you tried trying different thresholds? We have checked reproducibility at least 3 times, for different versions of PyTorch. Also, we have had this issue in the past (#14) and using a different threshold solved the problem. Best

ahmedhshahin commented 5 years ago

Hello, Yes I have tried different thresholds and could produce good results. My point is: with dextr_coco model you got 87.8 but with threshold 0.8 I got an accuracy of 90.9! Do you have any idea about why did this happen? best