help!!!RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

[2662007798]

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin /home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so False /home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib')} warn(msg) /home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /usr/local/cuda/lib did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs detected: 7.5 CUDA SETUP: Detected CUDA version 121 CUDA SETUP: Loading binary /home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so... /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda121.so) CUDA SETUP: Something unexpected happened. Please compile from source: git clone git@github.com:TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=121 python setup.py install Traceback (most recent call last): File "/home/yed_dev/qlora/qlora.py", line 15, in import bitsandbytes as bnb File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in from . import cuda_setup, utils, research File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 1, in from . import nn File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/optim/init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "/home/yed_dev/.local/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

python qlora.py --model_name_or_path /dev/shm/chatglm-6b error

[2662007798]

Please help me understand why such an error occurred?

[KJ-Waller]

Maybe give this solution a shot?

[kizunasunhy]

Upgrade to bitsandbytes==0.41.0 worked for me

[lunaryan]

bitsandbytes==0.41.0

Thank you sooo much! After searching under the bitsandbytes repo, by changing the libcuda.so, the LD_LIBRARY_PATH, by cp a bitsandbytes_cuda118.so to bitsandbytes_cpu.so, uninstall and reinstall pytorch again and again, your solution finally prevents the cuda warnings!

---

Despite no warnings and errors, it seems to still use cpu...