search

kjunelee / MetaOptNet

Meta-Learning with Differentiable Convex Optimization (CVPR 2019 Oral)
Apache License 2.0
524 stars 97 forks source link

Reproducability about ProtoNet on mini-ImageNet #25

Closed hankook closed 4 years ago

hankook commented 5 years ago

I am currently trying to reproduce results on mini-ImageNet using ProtoNet based on your code.

However, when I run the below script for training ProtoNet, it achieves 57% on the test set, but the reported accuracy is 59%.

python train.py --gpu 0,1,2,3 --save-path "./experiments/miniImageNet_MetaOptNet_ProtoNet" --train-shot 1 --val-shot 1 \
--head ProtoNet --network ResNet --dataset miniImageNet --eps 0.1

I wonder how to achieve the reported accuracy.

If you don't mind, can you tell me how you train ProtoNet on mini-ImageNet?

kjunelee commented 5 years ago

Sorry for the late reply. At the moment, I don't have a GPU server to run experiments on miniImageNet. Let me get back to this in few days.

kjunelee commented 4 years ago

Thanks for your interest in our work!

As mentioned in #8, each meta-training run can result in slightly different result. It seems like the result of both ProtoNet and MetaOptNet can vary across different environments. I experienced similar issues with many other few-shot learning algorithms.

The message of our paper is about the gap between non-parametric base learners and parametric base learners, and I believe that the gap should exist within the same environment.