ModuleNotFoundError: No module named 'oneccl_bindings_for_pytorch'

[beam-magnum]

I've encountered this issue when trying to build a chatbot using a python file, here's my code, copied from jupyter notebook:

from intel_extension_for_transformers.neural_chat import PipelineConfig
from intel_extension_for_transformers.neural_chat import build_chatbot
from intel_extension_for_transformers.neural_chat import plugins
from intel_extension_for_transformers.transformers import RtnConfig

plugins.retrieval.enable=True
plugins.retrieval.args['embedding_model'] = "ZhipuAI/chatglm3-6b"
plugins.retrieval.args["input_path"]="xxx"

config = PipelineConfig(model_name_or_path='AI-ModelScope/bge-base-zh-v1.5',
plugins=plugins,
optimization_config=RtnConfig(compute_dtype="int8",
weight_dtype="int4_fullrange"))

chatbot = build_chatbot(config)

Full traceback:

Traceback:
File "/opt/conda/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 600, in _run_script
    exec(code, module.__dict__)
File "/tmp/_MEIVqm4a2/studio_file/PROJECT/app.py", line 6, in <module>
    from intel_extension_for_transformers.transformers import RtnConfig
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/__init__.py", line 59, in <module>
    from .modeling import (
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/modeling/__init__.py", line 21, in <module>
    from .modeling_auto import (AutoModel, AutoModelForCausalLM,
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/modeling/modeling_auto.py", line 67, in <module>
    from ..llm.quantization.utils import (
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/llm/quantization/utils.py", line 37, in <module>
    import intel_extension_for_pytorch as ipex
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/__init__.py", line 122, in <module>
    from . import xpu
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/xpu/__init__.py", line 20, in <module>
    from .utils import *
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/xpu/utils.py", line 6, in <module>
    from .. import frontend
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/frontend.py", line 9, in <module>
    from .nn import utils
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/nn/__init__.py", line 1, in <module>
    from .modules import FrozenBatchNorm2d
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/nn/modules/__init__.py", line 8, in <module>
    from ...cpu.nn.linear_fuse_eltwise import IPEXLinearEltwise
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/cpu/nn/linear_fuse_eltwise.py", line 3, in <module>
    from intel_extension_for_pytorch.nn.utils._weight_prepack import (
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/nn/utils/__init__.py", line 1, in <module>
    from intel_extension_for_pytorch.nn.utils import _weight_prepack
File "/opt/conda/lib/python3.10/site-packages/intel_extension_for_pytorch/nn/utils/_weight_prepack.py", line 94, in <module>
    from deepspeed import comm
File "/opt/conda/lib/python3.10/site-packages/deepspeed/__init__.py", line 21, in <module>
    from . import ops
File "/opt/conda/lib/python3.10/site-packages/deepspeed/ops/__init__.py", line 6, in <module>
    from . import adam
File "/opt/conda/lib/python3.10/site-packages/deepspeed/ops/adam/__init__.py", line 6, in <module>
    from .cpu_adam import DeepSpeedCPUAdam
File "/opt/conda/lib/python3.10/site-packages/deepspeed/ops/adam/cpu_adam.py", line 8, in <module>
    from deepspeed.utils import logger
File "/opt/conda/lib/python3.10/site-packages/deepspeed/utils/__init__.py", line 10, in <module>
    from .groups import *
File "/opt/conda/lib/python3.10/site-packages/deepspeed/utils/groups.py", line 28, in <module>
    from deepspeed import comm as dist
File "/opt/conda/lib/python3.10/site-packages/deepspeed/comm/__init__.py", line 7, in <module>
    from .comm import *
File "/opt/conda/lib/python3.10/site-packages/deepspeed/comm/comm.py", line 34, in <module>
    from deepspeed.utils import timer, get_caller_func
File "/opt/conda/lib/python3.10/site-packages/deepspeed/utils/timer.py", line 31, in <module>
    class CudaEventTimer(object):
File "/opt/conda/lib/python3.10/site-packages/deepspeed/utils/timer.py", line 33, in CudaEventTimer
    def __init__(self, start_event: get_accelerator().Event, end_event: get_accelerator().Event):
File "/opt/conda/lib/python3.10/site-packages/deepspeed/accelerator/real_accelerator.py", line 145, in get_accelerator
    from .cpu_accelerator import CPU_Accelerator
File "/opt/conda/lib/python3.10/site-packages/deepspeed/accelerator/cpu_accelerator.py", line 8, in <module>
    import oneccl_bindings_for_pytorch  # noqa: F401 # type: ignore

GPT-4o tells me there's something wrong with intel-extension-for-pytorch but I can not find out why...

[beam-magnum]

I add this to my requirements.txt:

oneccl_bind_pt
--extra-index-url https://pytorch-extension.intel.com/release-whl/stable/cpu/us/

according to https://github.com/intel/torch-ccl, and it works.

[PenghuiCheng]

@beam-magnum , did you install intel_extension_for_transformers/neural_chat/requirements_xpu.txt, I think the oneccl_bindings_for_pytorch will be installed automaticly when you installed requirements_xpu.txt. It should be included in intel-extention-for-pytorch.

[beam-magnum]

@beam-magnum , did you install intel_extension_for_transformers/neural_chat/requirements_xpu.txt, I think the oneccl_bindings_for_pytorch will be installed automaticly when you installed requirements_xpu.txt. It should be included in intel-extention-for-pytorch.

but I'm deploying on cpus, my requirements.txt satisfy neural_chat/requirements.txt and neural_chat/requirements_cpu.txt. And they bath have intel-extension-for-pytorch: intel-extension-for-pytorch==2.1.30+xpu intel_extension_for_pytorch==2.3.0

[PenghuiCheng]

@beam-magnum , did you install intel_extension_for_transformers/neural_chat/requirements_xpu.txt, I think the oneccl_bindings_for_pytorch will be installed automaticly when you installed requirements_xpu.txt. It should be included in intel-extention-for-pytorch.

but I'm deploying on cpus, my requirements.txt satisfy neural_chat/requirements.txt and neural_chat/requirements_cpu.txt. And they bath have intel-extension-for-pytorch: intel-extension-for-pytorch==2.1.30+xpu intel_extension_for_pytorch==2.3.0

Yes, I see, it seems ipex for CPU also depends on oneccl_bind_pt now, it is a new feature. I will add this package in requirements_cpu.txt.