search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.27k stars 370 forks source link

open llama 1B #91

Open xinfu607 opened 10 months ago

xinfu607 commented 10 months ago

Hi, thanks a lot for your open resource. Please let me know if open llama 1B in the schedule.

young-geng commented 10 months ago

We currently don't have plans for 1B, as 1B might be too small to get interesting behavior out of language models.

Green-Sky commented 10 months ago

someone started to train a 1.1B llama, with gqa https://github.com/jzhang38/TinyLlama

kha84 commented 9 months ago

@xinfu607 look up phi-1 and phi-1.5 models