tim-learn / SHOT

code released for our ICML 2020 paper "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation"
MIT License
437 stars 77 forks source link

fix bugs in digit codes #9

Closed wengzejia1 closed 3 years ago

wengzejia1 commented 3 years ago

Hello, your work is excellent. I'm very interested in your work, so I ran your code and reproduced most of the experiment results. In the process of reproduction, I found three problems in the digital code.

  1. Before calling the 'obtain_label()' function, ”netB.eval()" is mistakenly written as "netF.eval()", resulting in the BN layer of bottleneck module mistakenly stays in the .train() state.
  2. In the "train_source()" function, you forgot to add "netF.train()", "netB.train()" and "netC.train()" in the end of the model validation process, so the mean and variance of the BN layers can not be updated after the first finished the first model validation.
  3. The "train_target()" function has the same problem as above. So Line of "netF.train()" and "netB.train()" should be added.

After fixing these bugs in the digit experiment code, the experiment results will be changed to some extent, especially the results of SHOT-IM method. The training process of SHOT-IM method will become stable and the score improvement is very obvious. The scores of SHOT-IM in my reproduction results are as follows.

SHOT-IM S->M: 99.0±0.0; U->M: 98.3±0.5; M->U: 98.1±0.1; Avg.: 98.5.

tim-learn commented 3 years ago

Hello, your work is excellent. I'm very interested in your work, so I ran your code and reproduced most of the experiment results. In the process of reproduction, I found three problems in the digital code.

  1. Before calling the 'obtain_label()' function, ”netB.eval()" is mistakenly written as "netF.eval()", resulting in the BN layer of bottleneck module mistakenly stays in the .train() state.
  2. In the "train_source()" function, you forgot to add "netF.train()", "netB.train()" and "netC.train()" in the end of the model validation process, so the mean and variance of the BN layers can not be updated after the first finished the first model validation.
  3. The "train_target()" function has the same problem as above. So Line of "netF.train()" and "netB.train()" should be added.

After fixing these bugs in the digit experiment code, the experiment results will be changed to some extent, especially the results of SHOT-IM method. The training process of SHOT-IM method will become stable and the score improvement is very obvious. The scores of SHOT-IM in my reproduction results are as follows.

SHOT-IM S->M: 99.0±0.0; U->M: 98.3±0.5; M->U: 98.1±0.1; Avg.: 98.5.

Thanks very much, Weng zejia! I will update the new results several days later.