search

lixiaotong97 / DSU

[ICLR 2022] Official pytorch implementation of "Uncertainty Modeling for Out-of-Distribution Generalization" in International Conference on Learning Representations (ICLR) 2022.
142 stars 16 forks source link

Could not implement the paper's results of semantic segmentation task #6

Closed jiangzhengkai closed 1 year ago

jiangzhengkai commented 2 years ago

Hi, I run the code to implement semantic segmentation results of GTA5->Cityscapes. The baseline result is 38.8, which is 1.8 higher than you report in the paper. The DSU result is 41.05, which is 2.0 lower than you report in the paper.

I follow the code and run the experiment on 4 V100.

baseline detail: image

DSU detail: image

I want to know how you report the result in the paper, whether it is a mean way or max way?

lixiaotong97 commented 2 years ago

Hi, Thanks for your interest in our work!

The segmentation experiment adopts the result of last checkpoint to report, which follows the setup of the repo pAdain. (Our corresponding checkpoints and logs have been provides in link. )

  1. It would be some fluctuations due to the user environment, you could try to change seeds and run more times.
  2. Besides, some fluctuations would come out in the last checkpoints and similar case also occurs in pAdain issue.

Best regards.