Closed June01 closed 11 months ago
Thank you for your reminder, we will release the conversion scripts for Falcon and InternLM at accessory/tools/convert
. For InternLM, it's pretty simple:
import torch
ori = torch.load("model_tp0_pp0.pt", map_location="cpu")
new_state_dict = {}
for key, val in ori.items():
key = key.replace("model.", "")
key = key.replace("blocks.", "layers.")
key = "llma." + key
new_state_dict[key] = val
new_state_dict = {"model": new_state_dict}
torch.save(new_state_dict, "consolidated.00-of-01.model.pth")
Thanks for the reply. It looks like the hugging face version is being updated continuously, and the model ends with .bin, so would you please show how to convert the .bin model to .pth? Thanks again. Btw, I am wondering is there any difference in training hugging face model and pytorch model?
Thanks for the reply. It looks like the hugging face version is being updated continuously, and the model ends with .bin, so would you please show how to convert the .bin model to .pth? Thanks again. Btw, I am wondering is there any difference in training hugging face model and pytorch model?
Unfortunately, at present, we don't have the conversion script from .bin
to .pth
. You may write it by yourself, it should not be complex.
To the best of my knowledge, InternLM is trained with their own codebase, and checkpoints are saved in the ".pt" format. The Huggingface .bin
version is then converted from .pt
file. Thus I think the conversion from .pt
to .pth
is enough to cope with most demands. Please let me know if I missed some important use cases necessitating the .bin
->.pth
conversion.
Dear authors, thanks for your wonderful work. I found out LLaMA2-Accessory supports InternLM now, which is very much appreciated. When I refer to the model zoo, there is only one version. However, I would like to play with the others, like internlm-chat-7b, could you please provide the scripts to convert the hf weights to .pth? Thanks in advance.