search

dvlab-research / LLaMA-VID

Official Implementation for LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Apache License 2.0
623 stars 40 forks source link

When running inference code on MSVD-QA,some error ocurred... #37

Closed CUCldyyyyy closed 5 months ago

CUCldyyyyy commented 6 months ago

hey!your work impressed me a lot and i wanna to run inference on MSVD-QA dataset following your guide,and first i do not sure about the effect of params 'api_key' and 'api_base',so i just omit them and meet a problem like below:

raise RuntimeError( RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback): [Errno 13] Permission denied: '//root/libcudart.so'

CUCldyyyyy commented 6 months ago

It seems a permission error and i totally follow your guide to prepare the structure and data.Following is the whole bug report :

Traceback (most recent call last): File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1099, in _get_module return importlib.import_module("." + module_name, self.name) File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 32, in from ...modeling_utils import PreTrainedModel File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/modeling_utils.py", line 38, in from .deepspeed import deepspeed_config, is_deepspeed_zero3_enabled File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/deepspeed.py", line 37, in from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/accelerate/init.py", line 3, in from .accelerator import Accelerator File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in from .utils import ( File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/accelerate/utils/init.py", line 131, in from .bnb import has_4bit_bnb_layers, load_and_quantize_model File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/accelerate/utils/bnb.py", line 42, in import bitsandbytes as bnb File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in from . import cuda_setup, utils, research File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 1, in from . import nn File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/research/nn/init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/optim/init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 13, in setup.run_cuda_setup() File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 120, in run_cuda_setup binary_name, cudart_path, cc, cuda_version_string = evaluate_cuda_setup() File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 337, in evaluate_cuda_setup cudart_path = determine_cuda_runtime_lib_path() File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 295, in determine_cuda_runtime_lib_path cuda_runtime_libs.update(find_cuda_lib_in(value)) File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 231, in find_cuda_lib_in return get_cuda_runtime_lib_paths( File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 217, in get_cuda_runtime_lib_paths if (path / libname).is_file(): File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/pathlib.py", line 1322, in is_file return S_ISREG(self.stat().st_mode) File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/pathlib.py", line 1097, in stat return self._accessor.stat(self, follow_symlinks=follow_symlinks) PermissionError: [Errno 13] Permission denied: '//root/libcudart.so'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/LiDongyang/LDY/LLaMA-VID/llamavid/eval/model_msvd_qa.py", line 4, in from llamavid.constants import IMAGE_TOKEN_INDEX, DEFAULT_IMAGE_TOKEN, DEFAULT_IM_START_TOKEN, DEFAULT_IM_END_TOKEN File "/home/LiDongyang/LDY/LLaMA-VID/llamavid/init.py", line 1, in from .model import LlavaLlamaAttForCausalLM File "/home/LiDongyang/LDY/LLaMA-VID/llamavid/model/init.py", line 1, in from .language_model.llava_llama_vid import LlavaLlamaAttForCausalLM File "/home/LiDongyang/LDY/LLaMA-VID/llamavid/model/language_model/llava_llama_vid.py", line 22, in from transformers import AutoConfig, AutoModelForCausalLM, \ File "", line 1075, in _handle_fromlist File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1090, in getattr value = getattr(module, name) File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1089, in getattr module = self._get_module(self._class_to_module[name]) File "/home/LiDongyang/anaconda3/envs/llamavid/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1101, in _get_module raise RuntimeError( RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback): [Errno 13] Permission denied: '//root/libcudart.so'

***Looking forward to your reply!

yanwei-li commented 6 months ago

Hi, api_key and api_base indicate the KEY and BASE (optional) of GPT3.5 that are used to evaluate the results. And the error seems to be the user does not have enough permission to load the package in this environment. Maybe you can try to reset permission or check whether have permission to load /root/libcudart.so.

CUCldyyyyy commented 6 months ago

I see，when I change to root and rerun pip and some missing packages can be installed.Thanks again!