error: unable to locate module 'read'

[loretoparisi]

On Mac M1, build

from read import BufReader, File
     ^
mojo: error: failed to parse the provided Mojo

version:

% mojo --version
mojo 0.4.0 (9e33b013)

[tairov]

hi @loretoparisi , are you sure you have cloned the whole repo? it still uses read module because for Mojo version < 0.4.0 there was no IO operations support. We have a PR for this to be merged soon #50 <- see this

[loretoparisi]

Yes thank you, it worked just copying the read folder's module. Out of the blue it worker for the smaller models 👍🏾 🥇 , but it crashed the whole mac with the TinyChat 😵‍💫 It seems that it went in OOM before freezing the Mac.

[tairov]

Yeah, we haven't tested TinyChat locally yet, probably need to improve the weights loading first..

[loretoparisi]

Yeah, we haven't tested TinyChat locally yet, probably need to improve the weights loading first..

maybe mmap and KV caching?

[tairov]

@loretoparisi mmap should be idea, I filed a Feature Request to mojo team.

Meanwhile, it looks tinyllama looks working on my M1 Max -- 32 GB

./llama2 ../models/tl-chat.bin -z ../models/tok_tl-chat.bin -t 0.0 -j 5 -i "Hello"
num parallel workers: 5  SIMD width: 8
checkpoint size:  4400767004 [ 4196 MB ] | n layers: 22 | vocab size: 32003
Hello
What is the best way to learn a new language?<|im_end|>
<|im_start|>assistant
The best way to learn a new language is to practice regularly, with a native speaker or a language exchange partner who can provide feedback and support. Here are some tips for learning a new language:

1. Practice regularly: The best way to improve your language skills is to practice regularly with a native speaker or a language exchange partner. This will help you build a solid foundation of vocabulary, grammar, and pronunciation.

2. Use a language exchange partner: Working with a language exchange partner can be a great way to learn a new language. Exchange partners can provide you with a native speaker to practice with, as well as help you with grammar and vocabulary.

3. Use a language learning app: There are many language learning apps available that can help you learn a new language. Some popular apps for learning a new language include Duolingo, Babbel, and Rosetta Stone.

4. Listen to native speakers: Listening to native speakers can help you improve your pronunciation and vocabulary. You can find native
achieved tok/s:  20.312251075354467

[loretoparisi]

@loretoparisi mmap should be idea, I filed a Feature Request to mojo team.

Meanwhile, it looks tinyllama looks working on my M1 Max -- 32 GB

./llama2 ../models/tl-chat.bin -z ../models/tok_tl-chat.bin -t 0.0 -j 5 -i "Hello"
num parallel workers: 5  SIMD width: 8
checkpoint size:  4400767004 [ 4196 MB ] | n layers: 22 | vocab size: 32003
Hello
What is the best way to learn a new language?<|im_end|>
<|im_start|>assistant
The best way to learn a new language is to practice regularly, with a native speaker or a language exchange partner who can provide feedback and support. Here are some tips for learning a new language:

1. Practice regularly: The best way to improve your language skills is to practice regularly with a native speaker or a language exchange partner. This will help you build a solid foundation of vocabulary, grammar, and pronunciation.

2. Use a language exchange partner: Working with a language exchange partner can be a great way to learn a new language. Exchange partners can provide you with a native speaker to practice with, as well as help you with grammar and vocabulary.

3. Use a language learning app: There are many language learning apps available that can help you learn a new language. Some popular apps for learning a new language include Duolingo, Babbel, and Rosetta Stone.

4. Listen to native speakers: Listening to native speakers can help you improve your pronunciation and vocabulary. You can find native
achieved tok/s:  20.312251075354467

thank you! Do you have the link to the Feature request, so I can support it! Thank you very much!

[tairov]

@loretoparisi here are 3 relevant Feature Requests I created https://github.com/modularml/mojo/issues/1134 https://github.com/modularml/mojo/issues/1135 https://github.com/modularml/mojo/issues/1130

[loretoparisi]

Thanks! Just shared https://x.com/loretoparisi/status/1718120443209523417?s=46&t=LFe0jyu2TCnvXZNEvnb71g