Unclear what trunc_normal init is.

facebookresearch / ClassyVision

An end-to-end PyTorch framework for image and video classification

https://classyvision.ai

MIT License

1.59k stars 278 forks source link

Unclear what trunc_normal init is. #683

Closed karttikeya closed 3 years ago

karttikeya commented 3 years ago

🐛 Bug

Vision Transformer uses torch.nn.init.trunc_normal_ (here) which is not yet available in torch.nn.init (in pytorch 1.4). Is this expected behavior?

mannatsingh commented 3 years ago

Ah yeah, you're right @karttikeya - this only would work with pytorch 1.7. To unblock yourself, you can upgrade to that. In the meantime, I will send a PR to fix this for older pytorch versions

mannatsingh commented 3 years ago

Fixed in https://github.com/facebookresearch/ClassyVision/pull/690