search

tairov / llama2.mojo

Inference Llama 2 in one file of pure 🔥
https://www.modular.com/blog/community-spotlight-how-i-built-llama2-by-aydyn-tairov
MIT License
2.09k stars 140 forks source link

question:Are these speed comparisons all in CPU mode? Can we add a comparison with GPU? #8

Closed xiezhipeng-git closed 1 year ago

xiezhipeng-git commented 1 year ago

Are these speed comparisons all in CPU mode? Can we add a comparison with GPU? Also, if you want to train, you want to use Mojo training. Is it necessary to add training related code in this way? Will rewriting be time-consuming?

tairov commented 1 year ago

Yes, all in CPU mode only. I would love to make it working on GPU.. But unfortunately Mojo doesn't support GPU execution yet.