Can't Reproduce MAML++ accuracy on Mini-Imagenet, In fact MAML outperforms it.

[RoozbehSanaei]

I wasn't able to replicate the paper's reported accuracy of approximately 50% for 1-shot learning using the present configuration file (mini-imagenet_maml++-mini-imagenet_1_2_0.01_48_5_0.json). I only achieved about 40%, which is below the MAML results from another open-source code (https://github.com/dragen1860/MAML-Pytorch). Have I made an error somewhere?"

[AntreasAntoniou]

That is certainly odd. The configuration file you refer to is for omniglot however.

The runner script for the experiment you refer to is in https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch/blob/master/experiment_scripts/mini-imagenet_maml%2B%2B-mini-imagenet_1_2_0.01_48_5_0_few_shot.sh

I can confirm that with the original paper I rerun my experiments multiple times on different machines, and at least back in November 2018 when I was working on this, I could reproduce the results -- but not just that, colleagues and students who used my code afterwards were also able to reproduce the results and build on my code.

[RoozbehSanaei]

Thank you for responding quickly.

I meant to type "mini-imagenet_maml++-mini-imagenet_1_2_0.01_48_5_0.json".

Do you mean "mini-imagenet_maml-mini-imagenet_1_2_0.01_48_5_0_few_shot.sh" should be used instead of "mini-imagenet_maml++-mini-imagenet_1_2_0.01_48_5_0_few_shot.sh"?

Is there anything more than just executing the bash script required to ensure it works correctly?

[AntreasAntoniou]

For the MAML++ model you need to run the maml++ config, and for MAML the maml.

[RoozbehSanaei]

It works correctly, my mistake!

[AntreasAntoniou]

Glad to hear that it all works well! :)