llama.cpp version llama-b2050-bin-win-avx2-x64
version: 2050 (19122117)
Windows 10
Running on AMD 3900x CPU
Command: server --threads 23 --ctx-size 16384 --mlock --model models\phind-codellama-34b-python-v1.Q4_K_M.gguf
(removing the --ctx-size 16384 --mlock parameters gets rid of the warning but, doesn't fix the error where it exits.)
I figured out that this is an error caused by a corrupt model. I downloaded using wget but included the ?download=true at the end of the URL which I think is where the error came from.
llama.cpp version llama-b2050-bin-win-avx2-x64 version: 2050 (19122117)
Windows 10 Running on AMD 3900x CPU Command:
server --threads 23 --ctx-size 16384 --mlock --model models\phind-codellama-34b-python-v1.Q4_K_M.gguf
(removing the--ctx-size 16384 --mlock
parameters gets rid of the warning but, doesn't fix the error where it exits.)Happens with all of these models:
https://huggingface.co/TheBloke/Phind-CodeLlama-34B-Python-v1-GGUF/resolve/main/phind-codellama-34b-python-v1.Q4_K_M.gguf?download=true
https://huggingface.co/TheBloke/CodeLlama-34B-Python-GGUF/resolve/main/codellama-34b-python.Q4_K_M.gguf?download=true
https://huggingface.co/senseable/Smaug-72B-v0.1-gguf/resolve/main/Smaug-72B-v0.1-q4_k_m.gguf?download=true
I'm probably doing something stupid or missing something. When I try to run these models, I get the following output and then the program exits: