search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.74k stars 1.27k forks source link

Text Generation WebUI issue #11618

Open wluo1007 opened 4 months ago

wluo1007 commented 4 months ago

openbmb/MiniCPM-2B-sft-bf16 on Text Generation WebUI issue on ubuntu22.04:

set up the webUI following similar steps as window described here: https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/webui_quickstart.md

when loading the model, error would show as below: Screenshot 2024-07-16 193211

sgwhat commented 4 months ago

hi @wluo1007 , we have reproduced your issue. We will inform you once we make progress.

sgwhat commented 4 months ago

Hi @wluo1007 , you may set export BIGDL_IMPORT_IPEX=0 before launching the webui.

wluo1007 commented 4 months ago

thanks for the quick response, after export BIGDL_IMPORT_IPEX=0, it worked.

niceTeen84 commented 4 months ago

Hi @sgwhat I have a question, webUI default bind adress is 127.0.0.1 how to change it?

sgwhat commented 4 months ago

@niceTeen84 , python server.py --load-in-4bit --listen --listen-host 0.0.0.0 --listen-port 8080