ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc

Apache License 2.0

6.65k stars 1.26k forks source link

ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent #10550

Open rmurtazi opened 7 months ago

rmurtazi commented 7 months ago

I tried to install IPEX-LLM using the following instruction: https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html

Installation works w/o any problems. When I try to do validation of installed packages I get an error:

(llm) gta@DUT7213ATSM:~$ python Python 3.9.19 (main, Mar 21 2024, 17:11:28) [GCC 11.2.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM Traceback (most recent call last): File "", line 1, in File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/ipex_llm/init.py", line 23, in from .optimize import optimize_model File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/ipex_llm/optimize.py", line 17, in import torch File "/home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/init.py", line 235, in from torch._C import * # noqa: F403 ImportError: /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent

yangw1234 commented 7 months ago

Hi @rmurtazi , could you run this script in your python environment and share the result? https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/scripts/env-check.sh

tsantra commented 7 months ago

Hi yangw1234 , I am also getting this same error.

tsantra commented 7 months ago

log.txt @yangw1234 attaching the log file which has the output of the env-check.sh script. It says IPEX is not installed. I have followed the following installation steps:

yangw1234 commented 7 months ago

@tsantra could you try pip list | grep torch?

tsantra commented 7 months ago

@yangw1234

yangw1234 commented 7 months ago

@tsantra could you try pip list | grep torch?

Hi @tsantra, I see your oneapi version is 2024.1 and current intel ipex/pytorch is built on openapi 2024.0, so I think this is the problem.

Intel just released 2024.1 recently so the installation instructions needs to be updated because it will install the latest oneapi version by default.

tsantra commented 7 months ago

@yangw1234 could you please give me the instruction to downgrade to 2024.0

qiuxin2012 commented 7 months ago

If you install oneapi using apt, you can:

sudo apt autoremove -y intel-basekit
sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

If you install oneapi using offline package, you can delete the 2024.1 version(just remove the oneapi folder), and install https://registrationcenter-download.intel.com/akdlm/IRC_NAS/163da6e4-56eb-4948-aba3-debcec61c064/l_BaseKit_p_2024.0.1.46_offline.sh

rmurtazi commented 7 months ago

@yangw1234 After downgrading to oneAPI 2024.0.1 I get the following: ython 3.9.19 (main, Mar 21 2024, 17:11:28) [GCC 11.2.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 2024-03-27 08:07:49,448 - INFO - intel_extension_for_pytorch auto imported

qiuxin2012 commented 7 months ago

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

qiuxin2012 commented 7 months ago

@yangw1234 After downgrading to oneAPI 2024.0.1 I get the following: ython 3.9.19 (main, Mar 21 2024, 17:11:28) [GCC 11.2.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

from ipex_llm.transformers import AutoModel, AutoModelForCausalLM /home/gta/miniconda3/envs/llm/lib/python3.9/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 2024-03-27 08:07:49,448 - INFO - intel_extension_for_pytorch auto imported

It's just some warnings, you can ignore them. From the INFO, it looks you have imported IPEX correctly.

jason-dai commented 7 months ago

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

@chtanch can you please verify?

hkvision commented 7 months ago

Sorry I just find sudo apt install intel-basekit=2024.0.1-43 is installing 2024.1 again, you should use below command:

sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
    intel-oneapi-compiler-cpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-compiler-dpcpp-eclipse-cfg=2024.0.2-49895 \
    intel-oneapi-diagnostics-utility=2024.0.0-49093 \
    intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
    intel-oneapi-mkl=2024.0.0-49656 \
    intel-oneapi-mkl-devel=2024.0.0-49656 \
    intel-oneapi-mpi=2021.11.0-49493 \
    intel-oneapi-mpi-devel=2021.11.0-49493 \
    intel-oneapi-tbb=2021.11.0-49513  \
    intel-oneapi-tbb-devel=2021.11.0-49513 \
    intel-oneapi-ccl=2021.11.2-5  \
    intel-oneapi-ccl-devel=2021.11.2-5 \
    intel-oneapi-dnnl-devel=2024.0.0-49521 \
    intel-oneapi-dnnl=2024.0.0-49521 \
    intel-oneapi-tcm-1.0=1.0.0-435

@chtanch can you please verify?

@NovTi Will help fix it.

NineMeowICT commented 6 months ago

Downgrading oneAPI is a fallback though. If you want to use the latest oneAPI. You can try to run the following command after the normal installation:

python -m pip install torch==2.1.0.post0 torchvision==0.16.0.post0 torchaudio==2.1.0.post0 intel-extension-for-pytorch==2.1.20+xpu oneccl_bind_pt==2.1.200+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

I knew this method from another issue https://github.com/pytorch/pytorch/issues/123097 describing a same problem.

ps: I searched this repo and found it seems that Intel hasn't updated its package requirements completely so ipex-llm will raise the error when using the new version of oneAPI. Come on, Intel, just fix it!

John-D-Boom commented 6 months ago

If you don't have sudo access, simply adding mkl==2024.0.0 to my environment.yml was enough to resolve the issue for now by downgrading mkl back to a compatible version.