search

learnables / learn2learn

A PyTorch Library for Meta-learning Research
http://learn2learn.net
MIT License
2.61k stars 350 forks source link

Why is learn2learn using truncated_normal_ as the weight initialization method? #336

Closed ATheCoder closed 2 years ago

ATheCoder commented 2 years ago

It seems like the xavier or the Kaiming which is used by PyTorch as a default is considered a good way to initialize the weights.

However, checking the code I realized you are changing that to a normal distribution.

Can I know the reason?

seba-1511 commented 2 years ago

It follows the original implementation of MAML. I don't think that makes too big of a difference.