search

yue-zhongqi / ifsl

[NeurIPS 2020] Released code for Interventional Few-Shot Learning
164 stars 23 forks source link

question about running this code with tieredwrn_1_ifsl.yaml in file SIB #7

Open Daryl003 opened 2 years ago

Daryl003 commented 2 years ago

hello authors! There is a problem while I'm running the code with config file tieredwrn_1_ifsl.yaml, an assertion error occurred. And it happened in class LinearDiag, networks.py. It shows that error occurred in “assert(X.dim()==2 and assert(X.dim()==2 and X.size(1)==self.weight.size(0)))”, and I finally find " (X.size(1)==self.weight.size(0)) = False", because the value of X.size(1) is 640 and the value of self.weight.size(0) is 512. Could you please give me some advises to solve this problem? Thanks a lot for your help!

yue-zhongqi commented 2 years ago

Hi, I think you are referring to SIB, right? I have updated the tieredwrn_1_ifsl.yaml config file (changing feat_dim from 512 to 640). Please try now and let me know.

Daryl003 commented 2 years ago

hello authors,  It is so kind of you for your help yesterday, and we have another problem now. While we're testing the module, we find that it randomly samples 2000 episodes which caused an unstable result. we try to run a couple of tests and average their results, but we still can't reproduce the result s of your paper. Therefore, we wonder if you have a specialized testing code to get a stable result. If you have that kind of code, could you please give us a copy of it? It would be very helpful! Thanks a lot for your help!

---Original--- From: @.> Date: Wed, Jul 28, 2021 09:00 AM To: @.>; Cc: @.**@.>; Subject: Re: [yue-zhongqi/ifsl] question about running this code with tieredwrn_1_ifsl.yaml in file SIB (#7)

Hi, I think you are referring to SIB, right? I have updated the tieredwrn_1_ifsl.yaml config file (changing feat_dim from 512 to 640). Please try now and let me know.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

yue-zhongqi commented 2 years ago

Hi Daryl,

No there is no specialized testing code. Can you share more details for your experiments?

Daryl003 commented 2 years ago

Hi Daryl,

No there is no specialized testing code. Can you share more details for your experiments?

Hello! Thanks for your reply! About experiments' details:

  1. yes, we downloaded the dataset from the provided link;
  2. we used the backbone from SimpleShot, and we will download it again from GoogleDrive.

And could you please tell us that how you get your final results in paper? we find when we test the best-result-model again, we can't get the same result while it was saved, e.g. we train with config file miniwrn_1_ifsl.yaml and get the model "netSIBBest75.553.pth". Then we use the same config file to test with the model "netSIBBest75.553.pth", and we get the finally result in 71%~72%. The results are very different. Therefore, we want to know your strategy for getting test results. Thanks a lot!