search

SCIR-HI / Huatuo-Llama-Med-Chinese

Repo for BenTsao [original name: HuaTuo (华驼)], Instruction-tuning Large Language Models with Chinese Medical Knowledge. 本草（原名：华驼）模型仓库，基于中文医学知识的大语言模型指令微调
Apache License 2.0
4.56k stars 454 forks source link

[BUG]运行infer.py cuda设置失败报错 #48

Closed woshicqy closed 1 year ago

woshicqy commented 1 year ago

请问一下，测试用的环境是什么啊。我是用的python3.10.11，安装完环境，cuda（版本为11.7）后运行infer.py python infer.py --base_model 'decapoda-research/llama-7b-hf' --lora_weights './lora-llama-med' --use_lora True --instruct_dir './data/infer.json' --prompt_template 'med_template'会报如下错误： Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/Users/cqy/.conda/envs/huatuo_cuda117/lib'), WindowsPath('C')} warn(msg) C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: C:\Users\cqy.conda\envs\huatuo_cuda117 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/127.0.0.1'), WindowsPath('7890'), WindowsPath('http')} warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')} warn(msg) CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine! C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)! warn(msg) C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library... warn(msg) CUDA SETUP: Loading binary C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so... argument of type 'WindowsPath' is not iterable CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected. CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig. CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following: CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc Traceback (most recent call last): File "E:\cqy-gpt\gpt-fine-tune\Huatuo-Llama-Med-Chinese\infer.py", line 8, in from peft import PeftModel File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\peft__init.py", line 22, in from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CONFIG_MAPPING, get_peft_config, get_peft_model File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\peft\mapping.py", line 16, in from .peft_model import ( File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\peft\peft_model.py", line 31, in from .tuners import ( File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\peft\tuners__init.py", line 21, in from .lora import LoraConfig, LoraModel File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\peft\tuners\lora.py", line 40, in import bitsandbytes as bnb File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\init.py", line 6, in from . import cuda_setup, utils, research File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\research\init__.py", line 1, in from . import nn File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\research\nn\init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\optim\init__.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "C:\Users\cqy.conda\envs\huatuo_cuda117\lib\site-packages\bitsandbytes\cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

请问这个问题要如何解决？感谢

s65b40 commented 1 year ago

应该是cuda环境的问题，需要配置cuda并添加环境变量