search

DeLightCMU / RSC

This is the official implementation of Self-Challenging Improves Cross-Domain Generalization, ECCV2020
BSD 2-Clause "Simplified" License
160 stars 18 forks source link

cannot reproduce results on PACS #5

Closed Johnzhangt closed 3 years ago

Johnzhangt commented 3 years ago

I run your code on PACS to reproduce your results in Table 6. I run three times, the results are:

Best val 0.965909, corresponding test 0.68847 - best test: 0.787732, best epoch: 18

Best val 0.967532, corresponding test 0.771698 - best test: 0.782642, best epoch: 25

Best val 0.962662, corresponding test 0.750573 - best test: 0.780606, best epoch: 17

The average result is 73.69, which is significantly lower than your results (80.85) in Table 6 for Res18.

I guess I may be wrong somewhere. How to reproduce results?

My environment is:

pytorch=1.1.0, torchvision=0.3.0.

Johnzhangt commented 3 years ago

If I do not use ImageNet pretrain model, I run the code for three times, the results are:

Best val 0.683442, corresponding test 0.443879 - best test: 0.471367, best epoch: 26

Best val 0.683442, corresponding test 0.370832 - best test: 0.404174, best epoch: 28

Best val 0.678571, corresponding test 0.382795 - best test: 0.484856, best epoch: 28

I am looking forward to your reply.

Justinhzy commented 3 years ago

Hi, could you run the code using my latest environment? please see #2 If you want to train your network without pretrained model, you may need other techniques which are not in the scope of this paper.