Closed loretoparisi closed 10 months ago
hi @loretoparisi , are you sure you have cloned the whole repo? it still uses read module because for Mojo version < 0.4.0 there was no IO operations support. We have a PR for this to be merged soon #50 <- see this
Yes thank you, it worked just copying the read
folder's module.
Out of the blue it worker for the smaller models 👍🏾 🥇 , but it crashed the whole mac with the TinyChat 😵💫
It seems that it went in OOM before freezing the Mac.
Yeah, we haven't tested TinyChat locally yet, probably need to improve the weights loading first..
Yeah, we haven't tested TinyChat locally yet, probably need to improve the weights loading first..
maybe mmap and KV caching?
@loretoparisi mmap
should be idea, I filed a Feature Request to mojo team.
Meanwhile, it looks tinyllama looks working on my M1 Max -- 32 GB
./llama2 ../models/tl-chat.bin -z ../models/tok_tl-chat.bin -t 0.0 -j 5 -i "Hello"
num parallel workers: 5 SIMD width: 8
checkpoint size: 4400767004 [ 4196 MB ] | n layers: 22 | vocab size: 32003
Hello
What is the best way to learn a new language?<|im_end|>
<|im_start|>assistant
The best way to learn a new language is to practice regularly, with a native speaker or a language exchange partner who can provide feedback and support. Here are some tips for learning a new language:
1. Practice regularly: The best way to improve your language skills is to practice regularly with a native speaker or a language exchange partner. This will help you build a solid foundation of vocabulary, grammar, and pronunciation.
2. Use a language exchange partner: Working with a language exchange partner can be a great way to learn a new language. Exchange partners can provide you with a native speaker to practice with, as well as help you with grammar and vocabulary.
3. Use a language learning app: There are many language learning apps available that can help you learn a new language. Some popular apps for learning a new language include Duolingo, Babbel, and Rosetta Stone.
4. Listen to native speakers: Listening to native speakers can help you improve your pronunciation and vocabulary. You can find native
achieved tok/s: 20.312251075354467
@loretoparisi
mmap
should be idea, I filed a Feature Request to mojo team.Meanwhile, it looks tinyllama looks working on my M1 Max -- 32 GB
./llama2 ../models/tl-chat.bin -z ../models/tok_tl-chat.bin -t 0.0 -j 5 -i "Hello" num parallel workers: 5 SIMD width: 8 checkpoint size: 4400767004 [ 4196 MB ] | n layers: 22 | vocab size: 32003 Hello What is the best way to learn a new language?<|im_end|> <|im_start|>assistant The best way to learn a new language is to practice regularly, with a native speaker or a language exchange partner who can provide feedback and support. Here are some tips for learning a new language: 1. Practice regularly: The best way to improve your language skills is to practice regularly with a native speaker or a language exchange partner. This will help you build a solid foundation of vocabulary, grammar, and pronunciation. 2. Use a language exchange partner: Working with a language exchange partner can be a great way to learn a new language. Exchange partners can provide you with a native speaker to practice with, as well as help you with grammar and vocabulary. 3. Use a language learning app: There are many language learning apps available that can help you learn a new language. Some popular apps for learning a new language include Duolingo, Babbel, and Rosetta Stone. 4. Listen to native speakers: Listening to native speakers can help you improve your pronunciation and vocabulary. You can find native achieved tok/s: 20.312251075354467
thank you! Do you have the link to the Feature request, so I can support it! Thank you very much!
@loretoparisi here are 3 relevant Feature Requests I created https://github.com/modularml/mojo/issues/1134 https://github.com/modularml/mojo/issues/1135 https://github.com/modularml/mojo/issues/1130
On Mac M1, build
version: