search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.27k stars 370 forks source link

13B V2 model planned? #79

Open tmostak opened 11 months ago

tmostak commented 11 months ago

Thank you for all your work on this project, it's really great to have a fully OSS Llama backbone.

I was excited to see the V2 version of the models with the original Llama tokenizer, and found that using the 7B model, performance (measured by perplexity) was indeed improved over the V1 model.

Are there plans to train a V2 version of the 13B model? If so, any idea of an ETA for that?

imoneoi commented 11 months ago

Also excited to see V2 13B! Better with coding + 8192 native context length