search

microsoft / LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs
https://aka.ms/GeneralAI
MIT License
3.71k stars 283 forks source link

[MiniLLM] Llama weights conversion #283

Open aaab8b opened 1 week ago

aaab8b commented 1 week ago

The checkpoints in the huggingface of Llama(13b,7b) seems cannot be directly loaded in the model when training MiniLLM since its not considered model parallelism. Is there any way to convert the weight to mp=4?

t1101675 commented 1 week ago

We provide a script to convert the mp of a checkpoint. You can try it.