search

Maknee / minigpt4.cpp

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
MIT License
549 stars 26 forks source link

Generate a bin file in Linux. #8

Open Rui-Hu01 opened 1 year ago

Rui-Hu01 commented 1 year ago

In the "Convert the model to ggml" section of the README.md, what is the "" parameter referring to? Following the instructions in MiniGPT4, I ended up with a folder containing the model weights. What specific file should I point to with this parameter?

Rui-Hu01 commented 1 year ago

In the "Convert the model to ggml" section of the README.md, what is the "" parameter referring to? Following the instructions in MiniGPT4, I ended up with a folder containing the model weights. What specific file should I point to with this parameter?

I made a mistake. convert.py should be under llama.cpp instead of minigpt4. However, there is now a new error : 'Exception: Vocab size mismatch (model has 32001, but MiniGPT-4/model/weight_7b/tokenizer.model has 32000). Most likely you are missing added_tokens.json (should be in MiniGPT-4/model/weight_7b).'. It worked fine in Minigpt-4. How should I handle this problem? What is missing?

Maknee commented 12 months ago

Add a added_tokens.json with id of 32001 (I think) in the folder that contains the vocab and model.

This is a problem with how pytorch loads the file and how minigpt4 loads it I believe. I didn't explore it in depth, but after adding the json file, that should fix it and convert.py should convert to the correct weights.