Status of chat model - Githubissues

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Apache License 2.0

7.31k stars 426 forks source link

Status of chat model #94

Closed galleon closed 7 months ago

galleon commented 7 months ago

It seems to me that PY007/TinyLlama-1.1B-Chat-v0.3 model weights were updated after the the 1.5T weights we issued.

I just want to check that these model.safetensors weights are basically the same than pytorch_model.bin or were obtained through a new fine tuning based on 1.5T. At least the README.md seems to indicate v03 is based on 1T.

If that is the case, is there some plans to release Chat-v0.4 ?

jzhang38 commented 7 months ago

Yes, the model.safetensors weights are basically the same as pytorch_model.bin for Chat-V0.3.

We will train and release Chat-v0.4 soon.

jzhang38 commented 7 months ago

https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v0.4