Closed xuedaobian closed 8 months ago
@xuedaobian There's something weird with this particular model that you tried, as I see there are multiple parts for each file, and I haven't seen GGUF model files like that before.
I think you better try to use this one instead: https://huggingface.co/kroonen/phi-2-GGUF/tree/main
thank you @giladgd ,i visited the link of you sent, it's useful!
Issue description
error loading model: unknown model architecture: ''
Expected Behavior
While I was browsing Reddit, I found a GGUF model of phi-2 trained by others,but when i download the model from the repository ,it can not run 。
the reddit page: https://www.reddit.com/r/LocalLLaMA/comments/18hnhd6/tutorial_how_to_run_phi2_locally_or_on_colab_for/?rdt=58275
the resposity addr: https://huggingface.co/radames/phi-2-quantized/tree/main
when i execution
npx --no node-llama-cpp chat --model ./models/model-v2-qk4.gguf
andnpx --no node-llama-cpp chat --model ./models/model-v2-q80.gguf
, the error appear.Actual Behavior
in this image, mistral-xxx is can run successful, it from thebloke
Steps to reproduce
i just run simple commands from getting started part, those included by npx --no node-llama-cpp chat --model
and
node index.js(code from getting stared part)
My Environment
node-llama-cpp
versionAdditional Context
i dont kow how to dstinguish which model can be run, its so hard
thank you receive
Relevant Features Used
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, and I know how to start.