Update transformer_from_scratch.py

aladdinpersson / Machine-Learning-Collection

A resource for learning about Machine learning & Deep Learning

https://www.youtube.com/c/AladdinPersson

MIT License

7.69k stars 2.7k forks source link

Closed jzbdota closed 1 year ago

jzbdota commented 1 year ago

At line 62, the scaling factor should be self.head_dim instead of self.embed_size