haebeom-lee
commented
3 years ago Hi, thank you for having interest in our paper! I'm terribly sorry for the late reply. I completely forgot to answer.
Yes, you are correct that the usage of batch normalization is transductive here, meaning that the neighbor query examples are used at meta-testing stage, which may not be allowed in practice. And this is how the original MAML code has been implemented (see https://github.com/cbfinn/maml). Unfortunately, it is not so trivial how to correctly implement the batch normalization in MAML framework, maybe because the batch statistics keep evolvoing during the inner-optimization process of MAML. For those reasons, we decided to follow the conventional transductive batch normalization scheme used in the original MAML code, and also made sure that all the other baselines in our paper follow exactly the same setup.
I suggest you to read the following paper regarding this issue:
- Bronskill et al., TaskNorm: Rethinking Batch Normalization for Meta-Learning, ICML 2020 This paper formally define this issue and propose a novel BN scheme that can solve the problem. I hope this paper can help you a little bit.
Best regards, Hae Beom Lee.
Hi, thank you for sharing your code.
In the conv_block() function in layers.py, tf.contrib.layers.batch_norm() is used without the is_training flag modification.
Since the flag controls to update the running means, the normalization layers collect query data information during the meta test stage.
Could you explain about the batch normalization layer in your code? Thank you.