Closed Hongbin98 closed 1 year ago
You can just replace the luodian/xxxxx
to your local folder path like /home/xxxx
.
note that it's a folder path (Huggingface format model usually needs a folder to load, it would load the checkpoints inside the folder according to "config.json" inside the foler).
You can just replace the
luodian/xxxxx
to your local folder path like/home/xxxx
.note that it's a folder path (Huggingface format model usually needs a folder to load, it would load the checkpoints inside the folder according to "config.json" inside the foler).
I ever tried to do this, but got an error.
running: model = OtterForConditionalGeneration.from_pretrained("./Otter-main/OTTER-MPT7B-densecaption", device_map="auto")
. And there are files in this output dir, including 'config.json', 'final_weights.pt' and others.
Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory ./Otter-main/OTTER-MPT7B-densecaption.
It may be a problem in folder structure.
Inside the foler there should be a "config.json" and multiple "pytorch-0001/0004.bin"
You can check our huggingface repo to see the hf format folder.
You may check if you correctly saved your model in correct format.
There's a --save_hf_model
option for saving correct hf model.
You may also refer the https://github.com/Luodian/Otter/blob/main/otter/flamingo_pt2otter_hf.py
This script is used to convert a checkpoint final_weights.pt
to a correct hf format folder.
The OtterForConditionalGeneration
is a standard way to launch hf model, it takes the hf model folder as input~
Following the suggestion of @Luodian, I have solved this issue.
If anyone following the recommended training command to train otter and cannot load the checkpoint, I hope the following steps could offer you a hand~
converting_otter_pt_to_hf.py
to convert your local weight to the hugging face type, and save in a new directory, like ./OTTER-MPT7B-densecaption-tohf
OtterForConditionalGeneration.from_pretrained()
the of the demo file, i.e., luodian/xxxxx
to ./OTTER-MPT7B-densecaption-tohf
In my opinion, a better choice to solve this issue is to add the --save_hf_model
option when you train otter.
evaluation
(evaluation result, performance of Otter etc.),For example, the model is loaded by the following command from hugging face:
model = OtterForConditionalGeneration.from_pretrained("luodian/OTTER-9B-LA-InContext", device_map="sequential", **precision)
After training, I got my weights located on './Otter-main/OTTER-MPT7B-densecaption'.
My question is how to load my local weight and perform the otter demo? --I guess like
model = TempModel(); model.load_state_dict(torch.load(file_path))
Also, I would like to show my gratitude for your reply on recent days and cite your research in the future :)