search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.49k stars 1.24k forks source link

accelerate==0.23.0 should be installed for CPU QLoRA #9401

Closed Uxito-Ada closed 9 months ago

Uxito-Ada commented 10 months ago

Otherwise I encountered the below on SPR:

cannot import name 'dispatch_model' from partially initialized module 'accelerate' (most likely due to a circular import) (/root/anaconda3/envs/heyang-lora/lib/python3.9/site-packages/accelerate/__init__.py)
hzjane commented 10 months ago

This default accelerate version is 0.21.0. And i will update its version if I encounter no other problems. How did you encounter this problem?

Uxito-Ada commented 10 months ago

I encountered this issue again, when I run on SPR 01 after install, accelerate occurs the following bug:

Traceback (most recent call last):
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1282, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/transformers/integrations/peft.py", line 29, in <module>
    from accelerate import dispatch_model
ImportError: cannot import name 'dispatch_model' from partially initialized module 'accelerate' (most likely due to a circular import) (/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/heyang/deepspeed-finetune/qlora_finetuning_cpu.py", line 23, in <module>
    from peft import LoraConfig
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/peft/auto.py", line 30, in <module>
    from .config import PeftConfig
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/peft/utils/__init__.py", line 22, in <module>
    from .other import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/peft/utils/other.py", line 21, in <module>
    import accelerate
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/utils/__init__.py", line 133, in <module>
    from .launch import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/utils/launch.py", line 33, in <module>
    from ..utils.other import is_port_in_use, merge_dicts
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/utils/other.py", line 30, in <module>
    from deepspeed import DeepSpeedEngine
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/__init__.py", line 21, in <module>
    from . import ops
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/ops/__init__.py", line 6, in <module>
    from . import adam
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/ops/adam/__init__.py", line 6, in <module>
    from .cpu_adam import DeepSpeedCPUAdam
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/ops/adam/cpu_adam.py", line 8, in <module>
    from deepspeed.utils import logger
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/utils/__init__.py", line 10, in <module>
    from .groups import *
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/utils/groups.py", line 28, in <module>
    from deepspeed import comm as dist
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/comm/__init__.py", line 7, in <module>
    from .comm import *
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/comm/comm.py", line 34, in <module>
    from deepspeed.utils import timer, get_caller_func
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/utils/timer.py", line 31, in <module>
    class CudaEventTimer(object):
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/utils/timer.py", line 33, in CudaEventTimer
    def __init__(self, start_event: get_accelerator().Event, end_event: get_accelerator().Event):
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/deepspeed/accelerator/real_accelerator.py", line 110, in get_accelerator
    import intel_extension_for_pytorch  # noqa: F401,F811 # type: ignore
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/__init__.py", line 91, in <module>
    from . import xpu
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/xpu/__init__.py", line 20, in <module>
    from .utils import *
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/xpu/utils.py", line 6, in <module>
    from .. import frontend
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/frontend.py", line 10, in <module>
    from .nn import utils
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/__init__.py", line 1, in <module>
    from .modules import FrozenBatchNorm2d
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/modules/__init__.py", line 6, in <module>
    from ...cpu.nn.linear_fuse_eltwise import IPEXLinearEltwise
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/cpu/nn/linear_fuse_eltwise.py", line 3, in <module>
    from intel_extension_for_pytorch.nn.utils._weight_prepack import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/utils/__init__.py", line 1, in <module>
    from intel_extension_for_pytorch.nn.utils import _weight_prepack
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/utils/_weight_prepack.py", line 8, in <module>
    from intel_extension_for_pytorch.cpu.tpp.utils.blocked_layout import (
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/cpu/tpp/__init__.py", line 3, in <module>
    from . import fused_bert
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/intel_extension_for_pytorch/cpu/tpp/fused_bert.py", line 16, in <module>
    from transformers.modeling_utils import apply_chunking_to_forward
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/transformers/modeling_utils.py", line 40, in <module>
    from .integrations import PeftAdapterMixin, deepspeed_config, is_deepspeed_zero3_enabled
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1284, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.integrations.peft because of the following error (look up to see its traceback):
cannot import name 'dispatch_model' from partially initialized module 'accelerate' (most likely due to a circular import) (/root/anaconda3/envs/ziteng-llm/lib/python3.9/site-packages/accelerate/__init__.py)

upgrade from 0.21.0 to 0.23.0 can solve it. @hzjane

Uxito-Ada commented 9 months ago

fix in #9506