Yikai-Wang / ICI-FSL

This repository contains the code for Instance Credibility Inference (ICI).
82 stars 22 forks source link

About the experimental results on the MiniImageNet dataset #11

Closed lampardwk closed 3 years ago

lampardwk commented 3 years ago

Thank you very much for sharing the code. I ran the code and found that the accuracy of both 1-shot and 5-shot in the MiniImageNet dataset(semi) was 1~2% higher than that reported in the paper. I checked the config.py file and found some differences. In the paper, it was mentioned that the training epoch is 120(decayed by 10 after every 30 epochs) and test episodes is 600, but in config.py, the training epoch was 90(began to decay from 60 epoch,i.e.,(60, 0.1), (70, 0.006), (80, 0.0012), (90, 0.00024)) and test episodes is 2000. Are these the reasons for the inconsistencies? Thanks in advance.

Yikai-Wang commented 3 years ago

Thanks for the attention to our work. Our work of ICI V2 is in still under review. The code at this stage is a modified version. In this version, we used a new training strategy, so we got a better result. The updated manuscript will be uploaded to arxiv soon. In the test, we chose 2000 episodes because this is a more popular setting recently. In our experiment, the number of episodes only has a small effect on the confidence interval, and has no significant effect on the average accuracy.

lampardwk commented 3 years ago

Got it! Thanks for your help!