[Usage] Failed to run the model

[hjsg1010]

Describe the issue

Issue: RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback): 'NoneType' object is not subscriptable

Command:

python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path /home/model/llava/llava-336px-pretrain-vicuna-13b-v1.3 --model-base /home/model/llava/vicuna_13b_v1.3

Log:

/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cadam32bit_grad_fp32
Error: Attempting to get amgpu ISA Details 'NoneType' object has no attribute 'group'
Error: Attempting to get amgpu ISA Details 'NoneType' object has no attribute 'group'
Traceback (most recent call last):
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1099, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/home/.local/share/conda/envs/llava/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 32, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/modeling_utils.py", line 38, in <module>
    from .deepspeed import deepspeed_config, is_deepspeed_zero3_enabled
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/deepspeed.py", line 37, in <module>
    from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/accelerate/utils/__init__.py", line 131, in <module>
    from .bnb import has_4bit_bnb_layers, load_and_quantize_model
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/accelerate/utils/bnb.py", line 42, in <module>
    import bitsandbytes as bnb
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 16, in <module>
    from .nn import modules
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/nn/__init__.py", line 6, in <module>
    from .triton_based_modules import SwitchBackLinear, SwitchBackLinearGlobal, SwitchBackLinearVectorwise, StandardLinear
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/nn/triton_based_modules.py", line 8, in <module>
    from bitsandbytes.triton.dequantize_rowwise import dequantize_rowwise
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/bitsandbytes/triton/dequantize_rowwise.py", line 10, in <module>
    import triton
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/triton/__init__.py", line 20, in <module>
    from .runtime import (
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/triton/runtime/__init__.py", line 1, in <module>
    from .autotuner import Config, Heuristics, autotune, heuristics
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/triton/runtime/autotuner.py", line 7, in <module>
    from ..compiler import OutOfResources
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/triton/compiler.py", line 1895, in <module>
    @static_vars(amdgcn_bitcode_paths = _get_amdgcn_bitcode_paths())
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/triton/compiler.py", line 1874, in _get_amdgcn_bitcode_paths
    gfx_arch = _get_amdgcn_bitcode_paths.discovered_gfx_arch_fulldetails[1]
TypeError: 'NoneType' object is not subscriptable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.local/share/conda/envs/llava/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/.local/share/conda/envs/llava/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/home/LLaVA-main/llava/__init__.py", line 1, in <module>
    from .model import LlavaLlamaForCausalLM
  File "/home/LLaVA-main/llava/model/__init__.py", line 1, in <module>
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
  File "/home/LLaVA-main/llava/model/language_model/llava_llama.py", line 22, in <module>
    from transformers import AutoConfig, AutoModelForCausalLM, \
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1090, in __getattr__
    value = getattr(module, name)
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1089, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/.local/share/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1101, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
'NoneType' object is not subscriptable

I'm loading model from local files. so I changed Launch a model worker (LoRA weights, unmerged) command to upper command.

please let me know if there is problem thx for read my question and sorry for if it is silly problem

[hjsg1010]

I got same error from

from transformers import LlamaModel

this line.

I tried reinstall libraries, but still get same error

[hjsg1010]

It's weird.. when I try at pip enviorment (not conda virtual enviorment) it works.. It is not repo issues so I will try it to fix my self. sorry for bothering