search

yinboc / few-shot-meta-baseline

Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning, in ICCV 2021
MIT License
615 stars 107 forks source link

Question about the meta-test. #33

Closed z1358 closed 3 years ago

z1358 commented 3 years ago

Dear author, I have some questions about the meta-test stage. In your paper, you apply consistent sampling and the same 800 testing tasks are sampled for estimating the performence. But in the test_few_shot.py, I find the random seed is fixed as 0 and the default value of the parameter test-epochs is 10. Do you mean runing the same 800 testing tasks for ten times and computing the average acc? If so, since the meta-test stage is non-parametric, why different test-epochs get different acc? Looking forward to your reply！

MohamedAfham commented 3 years ago

Hi, I have the same question as @z1358. Can you please let me know,

  1. Why multiple testing epochs are used during meta-testing stage?
  2. Since the seed is set to be 0 (np.random.seed(0)), why different test-epochs get different acc ?
  3. For each test-epoch, we get average accuracy with std. How is the final accuracy reported in the paper calculated from the set of test_epochs?

Looking forward to hear from you!

flcaus commented 7 months ago

Hi， I have the same question @z1358 @MohamedAfham ，have you solved it?