search

okuvshynov / slowllama

Finetune llama2-70b and codellama on MacBook Air without quantization
MIT License
431 stars 33 forks source link

Mojo 🔥? #9

Open oaustegard opened 8 months ago

oaustegard commented 8 months ago

Now that mojo is available for M1/M2 platforms, have you considered attempting this with mojo for improved performance? (Questionable as to how much I guess with all the shuffling to the ssd?)

https://www.modular.com/blog/mojo-is-now-available-on-mac

Here is a llama2 implementation: https://github.com/tairov/llama2.mojo

okuvshynov commented 8 months ago

Oh, this is nice, i was not aware of it. Let me take a look.