search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.45k stars 1.24k forks source link

Process exit when I use "Pyinstaller" to package and run the Python demo code #11379

Open grandxin opened 2 months ago

grandxin commented 2 months ago

when I use Pyinstaller to package and run the Python demo code, The .exe process exit when

model = model.to('xpu')

There is no problem running the demo directly using Python.

anyone know how to do?

hkvision commented 2 months ago

Sorry that we haven't tried Pyinstaller on XPU yet. Will notify you if we have verified this.

tjtanaa commented 1 month ago

I encounter the same issue with pyinstaller compiled executable. The program exited within AutoModelForCausalLM without any error. I can observed that the progress bar of loading the weight is complete. 

        self.model = AutoModelForCausalLM.from_pretrained(
            model_path, trust_remote_code=True
        ).to(self.device)

There is also this error when launching the intel_extension_for_pytorch.

intel_extension_for_pytorch\xpu\lazy_init.py when loading extension _C._initExtension() is causing the following error. _C is _C.cp310-win_amd64.pyd

 intel_extension_for_pytorch\xpu\lazy_init.py:80: UserWarning: XPU Device count is zero! (Triggered internally at C:/Users/arc/ruijie/2.1_RC3/python310/frameworks.ai.pytorch.ipex-gpu/csrc/gpu/runtime/Device.cpp:127.)
  _C._initExtension()

Steps to reproduce

Can be found at https://github.com/intel/intel-extension-for-pytorch/issues/674#issuecomment-2228114289