Is the 1T tokens completed?

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

Apache License 2.0

7.29k stars 372 forks source link

Is the 1T tokens completed? #36

Closed alimadelshin closed 1 year ago

alimadelshin commented 1 year ago

is the 1T training completed or is there any issues with the training?

young-geng commented 1 year ago

The 1T token training is done for the 7b model, but we are waiting for the 3b model to complete so we can release them together. The anticipated release is next Monday.

young-geng commented 1 year ago

We've just released the 1T token final version for the 3B and 7B models. We've also released a 600B token preview for the 13B model.