search

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
Apache License 2.0
7.71k stars 453 forks source link

Converting Saved Model Files to Hugging Face Transformers Format #60

Closed dtxwhzw closed 11 months ago

dtxwhzw commented 12 months ago

Hello, I have been using your pre-trained code, and I'm wondering how to convert the saved model files into the Hugging Face Transformers format, similar to the ones you upload to Hugging Face's repository?

ChaosCodes commented 12 months ago

You can check this scripts/convert_lit_checkpoint.py. Run the following command to convert the model ckpt from .pth into .bin format.

python scripts/convert_lit_checkpoint.py \
    --checkpoint_name your_checkpoint.pth \
    --out_dir destination_dir/ \
    --model_name tiny_LLaMA_1b
Juhywcy commented 6 months ago

@ChaosCodes sorry, I use the command to convert the model ckpt from .pth into .bin, but the param "out_dir" can't work. It is always saved in the same path as ckpt dir. How to solve this?