search

Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.68k stars 170 forks source link

Could you please provide script to convert huggingface InternLM model to .pth? #70

Closed June01 closed 11 months ago

June01 commented 11 months ago

Dear authors, thanks for your wonderful work. I found out LLaMA2-Accessory supports InternLM now, which is very much appreciated. When I refer to the model zoo, there is only one version. However, I would like to play with the others, like internlm-chat-7b, could you please provide the scripts to convert the hf weights to .pth? Thanks in advance.

ChrisLiu6 commented 11 months ago

Thank you for your reminder, we will release the conversion scripts for Falcon and InternLM at accessory/tools/convert. For InternLM, it's pretty simple:

import torch

ori = torch.load("model_tp0_pp0.pt", map_location="cpu")

new_state_dict = {}
for key, val in ori.items():
    key = key.replace("model.", "")
    key = key.replace("blocks.", "layers.")
    key = "llma." + key
    new_state_dict[key] = val

new_state_dict = {"model": new_state_dict}

torch.save(new_state_dict, "consolidated.00-of-01.model.pth")
June01 commented 11 months ago

Thanks for the reply. It looks like the hugging face version is being updated continuously, and the model ends with .bin, so would you please show how to convert the .bin model to .pth? Thanks again. Btw, I am wondering is there any difference in training hugging face model and pytorch model?

ChrisLiu6 commented 11 months ago

Thanks for the reply. It looks like the hugging face version is being updated continuously, and the model ends with .bin, so would you please show how to convert the .bin model to .pth? Thanks again. Btw, I am wondering is there any difference in training hugging face model and pytorch model?

Unfortunately, at present, we don't have the conversion script from .bin to .pth. You may write it by yourself, it should not be complex.

To the best of my knowledge, InternLM is trained with their own codebase, and checkpoints are saved in the ".pt" format. The Huggingface .bin version is then converted from .pt file. Thus I think the conversion from .pt to .pth is enough to cope with most demands. Please let me know if I missed some important use cases necessitating the .bin ->.pth conversion.