search

yunjey / pytorch-tutorial

PyTorch Tutorial for Deep Learning Researchers
MIT License
29.79k stars 8.03k forks source link

Tutorials for the using of Transformer layer #182

Open zwx8981 opened 5 years ago

zwx8981 commented 5 years ago

Hi, is there any plan to provide a tutorial of showing an example of employing the Transformer as an alternative of RNN for seq2seq task such as machine translation?

mdhasanai commented 5 years ago

I found this great tutorial to learn about Transformers. Link: (http://jalammar.github.io/illustrated-transformer/)

If you decided to implement in PyTorch than this link is going to help you. (http://nlp.seas.harvard.edu/2018/04/03/attention.html)

arushirai1 commented 7 months ago

Coming back to this! It would be cool to see transformers for video data and the categories suggested above... I could work on this if this hasn't been done. The links that have been shared were great for their time, but now there are Transformer-specific modules in Pytorch that would be good to demonstrate.