Error message: "llama.cpp error: 'done_getting_tensors: wrong number of tensors; expected 388, got 195'"

[jackNhat]

I converted the model into the llama.cpp's own format (.GGUF). And this model could run successful in colab like snapshot.

But when i loaded this model in LLM Studio or Jan, i got error message. Anyone met this problem? How to resolve this?

Many thanks!

[xtfocus]

Likely a duplicated of my opening issue #22

[datquocnguyen]

See: https://github.com/ggerganov/llama.cpp/pull/5638

[datquocnguyen]

PR was merged. It should work fine now.

[churongcon]

@datquocnguyen Not working

[001123]

Same issue

[datquocnguyen]

@001123 @churongcon Please could you try installing/compiling llama.cpp from this specific commit and then redo the conversion step (to create the .gguf file):

https://github.com/jordankanter/llama.cpp/tree/87a41f53ae3a01dde4c198df72cfb99ba2c9f586

git clone https://github.com/jordankanter/llama.cpp
git checkout 87a41f53ae3a01dde4c198df72cfb99ba2c9f586

[datquocnguyen]

@001123 @churongcon, please disregard my previous comment. I have found the reason. "llama.cpp" has been modified, resulting in a converted version of phogpt-4b being associated with 387 tensors, instead of the previous 388. To load it with the latest version of Studio, you can simply reconvert phogpt-4b using the latest "llama.cpp".

[datquocnguyen]

@001123 @churongcon please try https://huggingface.co/vinai/PhoGPT-4B-Chat-gguf