Unable to reproduce results

zbf1991 / RRM

Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach

94 stars 11 forks source link

Unable to reproduce results #7

Closed ErikStammes closed 4 years ago

ErikStammes commented 4 years ago

Hi,

Thanks for the great work. I'm trying to reproduce your results, but I can't seem to get close to the scores you report in the paper.

I have tried two settings:

The default parameters as given in the code. This results in ~56.2 mIoU without CRF and ~57.6 mIoU with CRF
With parameters as given in the paper (lower learning rate (0.001) and lower image resolution (321)). This results in ~55.4 without CRF and ~57.4 with CRF.

Other than those things, I have not made any changes. Any idea why my scores are much lower? Are there other parameters which I need to set?

Thanks!

zbf1991 commented 4 years ago

I am not sure why your result is lower, do you train from init or from the pre-trained cls model? could you try again using train_cls_weight and the pre-trained classification model because it is more stable. Sorry for not solving your question.

ErikStammes commented 4 years ago

I used train_from_init.py but that already uses pretrained weights, right? i.e. ilsvrc-cls_rna-a1_cls1000_ep-0001.params.

I'll try to use train_from_cls_weight now. What is the difference between train_from_cls_weight.py and train_from_cls_weight(ori).py?

zbf1991 commented 4 years ago

Yeah. init used your mentioned model weights. no difference between these two file. (ori) is just the older version. I will check init.py again. Maybe there is some mistake when I update it. Sorry for that.

ErikStammes commented 4 years ago

Alright, thanks for the clarification!

ErikStammes commented 4 years ago

I am now able to reproduce your results using train_from_cls_weight.py, thanks!