search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.68k stars 1.26k forks source link

Cannot run Gemma2 on Ollama with ipex-llm #11627

Closed mouse33333 closed 3 months ago

mouse33333 commented 3 months ago

Hi

I can run Llama3, Qwen or other models on Ollama with ipex-llm. However, I cannot run Gemm2 on Ollama with ipex-llm. It shows unknown model architecture: 'gemma2'. Does it mean this model is not supported by ipex-llma?

Here're the details: screenshot-20240722-101506

Thank you for your help in advance~

sunnsi commented 3 months ago

+1. Please update the supported version of Ollama. The current version is 0.2.7, however, Ipex-LLM only supports up to 0.1.39."

rollinsjw commented 3 months ago

Also love this idea, enjoying running models locally and would love to play with deepseekv2 and gemma2

rnwang04 commented 3 months ago

Hi all, Gemma2 is supported with ipex-llm[cpp] >= 2.1.0b20240724, you may try it again : )

mouse33333 commented 3 months ago

Hi all, Gemma2 is supported with ipex-llm[cpp] >= 2.1.0b20240724, you may try it again : )

It works now! Thank you for your support!