Open Smupk2778 opened 1 week ago
Can someone help me out with this??
I am assuming you downloaded the model off of Huggingface. And since LLaMA 2 is very old it could be that there was some breaking change between the software that the GGUF file was created with and the current master version. Does it work if you convert the model yourself using the latest master code?
Like I build this llama-cpp in the AIX 7.2 which is based on the big-endian system,so the problem is there is some difficulty in converting the files in my os , so I directly downloaded the quantised gguf models from hugging face .
@Smupk2778 Llama2 is ancient at this point, try using llama3. If you have problem with it try using, llama-server or llama.cui it will log the conversion too https://github.com/dspasyuk/llama.cui
Ohkk I will try it, Thanks for help.
What happened?
I am using the llama-2-7b-chat.Q4_K_M.gguf and trying to run it using llama-cpp but I am not getting the actual output .I am getting output as # , not as any string.
Name and Version
What operating system are you seeing the problem on?
Big-Endian(Unix like system)
Relevant log output