search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.27k stars 370 forks source link

Updates on 20b version? #73

Closed pikalover6 closed 1 year ago

pikalover6 commented 1 year ago

Do you plan on releasing the v1 20b model?

gjmulder commented 1 year ago

Do you mean the llama 33 billion weight model? If so, given it takes three times as long (twice the size and 1.4B tokens) to train as the 13b I suspect not.

pikalover6 commented 1 year ago

Do you mean the llama 33 billion weight model? If so, given it takes three times as long (twice the size and 1.4B tokens) to train as the 13b I suspect not.

No, the creators have said that they trained a 20b model on discord/Twitter.

young-geng commented 1 year ago

I think the 20B model is trained by Stability AI, and we are not involved in that, so I wouldn't consider that part of this project.