search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.36k stars 374 forks source link

Is open-llama use the same tokenizer (sentencepiece) just like Meta LLaMa? #42

Closed tiendung closed 1 year ago

tiendung commented 1 year ago

thanks for providing a great dropin replacement for meta llama :)

young-geng commented 1 year ago

Yes we are using sentence piece to keep the model maximally compatible with LLaMA