(PI_ERROR_DEVICE_NOT_FOUND) llama_load_model_from_file: exception loading model

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.

Apache License 2.0

6.5k stars 1.24k forks source link

(PI_ERROR_DEVICE_NOT_FOUND) llama_load_model_from_file: exception loading model #11609

Open sundeepChandhoke opened 2 months ago

sundeepChandhoke commented 2 months ago

I am getting this error after installing the IPEX_LLM and onAPI. I can successfully serve ollama and pull models. But when I try to run any model, I get this error.

sundeepChandhoke commented 2 months ago

I do not have any Intel GPU on this machine. So maybe it is expecting a GPU? I am trying to test AMX instructions on the 4th gen Xeon processor

sgwhat commented 2 months ago

Hi @sundeepChandhoke , yes ollama is expecting a gpu and it does not support running on a device without any gpu currently.