search

bitsandbytes-foundation / bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.
https://huggingface.co/docs/bitsandbytes/main/en/index
MIT License
6.04k stars 606 forks source link

CUDA Setup failed despite CUDA being Available :: NameError: name 'str2optimizer32bit' is not defined #1190

Open NavinKumarMNK opened 4 months ago

NavinKumarMNK commented 4 months ago

System Info

Platform : ppc64le (Power 9) OS: Ubuntu 22.04

root@68e4b050ea94:/app# python3 -m bitsandbytes
WARNING: BNB_CUDA_VERSION=122 environment variable detected; loading libbitsandbytes_cuda122_nocublaslt122.so.
This can be used to load a bitsandbytes version that is different from the PyTorch CUDA version.
If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION=
If you use the manual override make sure the right libcudart.so is in your LD_LIBRARY_PATH
For example by adding the following to your .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_cuda_dir/lib64

Could not find the bitsandbytes CUDA binary at PosixPath('/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/libbitsandbytes_cuda122_nocublaslt122.so')
Could not load bitsandbytes native library: /root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/cextension.py", line 109, in <module>
    lib = get_native_library()
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/cextension.py", line 96, in get_native_library
    dll = ct.cdll.LoadLibrary(str(binary_path))
  File "/root/miniconda3/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/root/miniconda3/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory

CUDA Setup failed despite CUDA being available. Please run the following command to get more information:

python -m bitsandbytes

Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
CUDA specs: CUDASpecs(highest_compute_capability=(7, 0), cuda_version_string='122', cuda_version_tuple=(12, 2))
PyTorch settings found: CUDA_VERSION=122, Highest Compute Capability: (7, 0).
WARNING: BNB_CUDA_VERSION=122 environment variable detected; loading libbitsandbytes_cuda122_nocublaslt122.so.
This can be used to load a bitsandbytes version that is different from the PyTorch CUDA version.
If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION=
If you use the manual override make sure the right libcudart.so is in your LD_LIBRARY_PATH
For example by adding the following to your .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_cuda_dir/lib64

Library not found: /root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/libbitsandbytes_cuda122_nocublaslt122.so. Maybe you need to compile it from source?
If you compiled from source, try again with `make CUDA_VERSION=DETECTED_CUDA_VERSION`,
for example, `make CUDA_VERSION=113`.

The CUDA version for the compile might depend on your conda install, if using conda.
Inspect CUDA version via `conda list | grep cuda`.
To manually override the PyTorch CUDA version please see: https://github.com/TimDettmers/bitsandbytes/blob/main/docs/source/nonpytorchcuda.mdx
WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
If you run into issues with 8-bit matmul, you can try 4-bit quantization:
https://huggingface.co/blog/4bit-transformers-bitsandbytes
The directory listed in your path is found to be non-existent: //developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux_ppc64le.run
Found duplicate CUDA runtime files (see below).

We select the PyTorch default CUDA runtime, which is 12.2,
but this might mismatch with the CUDA version that is needed for bitsandbytes.
To override this behavior set the `BNB_CUDA_VERSION=<version string, e.g. 122>` environmental variable.

For example, if you want to use the CUDA version 122,
    BNB_CUDA_VERSION=122 python ...

OR set the environmental variable in your .bashrc:
    export BNB_CUDA_VERSION=122

In the case of a manual override, make sure you set LD_LIBRARY_PATH, e.g.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2,
* Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so
* Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so.12
* Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so.12.2.53
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Checking that the library is importable and CUDA is callable...
Couldn't load the bitsandbytes library, likely due to missing binaries.
Please ensure bitsandbytes is properly installed.

For source installations, compile the binaries with `cmake -DCOMPUTE_BACKEND=cuda -S .`.
See the documentation for more details if needed.

Trying a simple check anyway, but this will likely fail...
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/diagnostics/main.py", line 66, in main
    sanity_check()
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/diagnostics/main.py", line 40, in sanity_check
    adam.step()
  File "/root/miniconda3/lib/python3.10/site-packages/torch/optim/optimizer.py", line 373, in wrapper
    out = func(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/optim/optimizer.py", line 287, in step
    self.update_step(group, p, gindex, pindex)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/optim/optimizer.py", line 496, in update_step
    F.optimizer_update_32bit(
  File "/root/miniconda3/lib/python3.10/site-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-ppc64le.egg/bitsandbytes/functional.py", line 1584, in optimizer_update_32bit
    optim_func = str2optimizer32bit[optimizer_name][0]
NameError: name 'str2optimizer32bit' is not defined
Above we output some debug information.
Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose
WARNING: Please be sure to sanitize sensitive info from the output before posting it.

Reproduction

python3 -m bitsandbytes

Expected behavior

Expected not to produce any error.

shigabeev commented 4 months ago

I'm having the same issue

root@1c1b2c566ff1:/workspace/ilya_png# python -m bitsandbytes
Could not find the bitsandbytes CUDA binary at PosixPath('/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cuda121.so')
Could not load bitsandbytes native library: /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/cextension.py", line 109, in <module>
    lib = get_native_library()
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/cextension.py", line 96, in get_native_library
    dll = ct.cdll.LoadLibrary(str(binary_path))
  File "/usr/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory

CUDA Setup failed despite CUDA being available. Please run the following command to get more information:

python -m bitsandbytes

Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
CUDA specs: CUDASpecs(highest_compute_capability=(8, 9), cuda_version_string='121', cuda_version_tuple=(12, 1))
PyTorch settings found: CUDA_VERSION=121, Highest Compute Capability: (8, 9).
Library not found: /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cuda121.so. Maybe you need to compile it from source?
If you compiled from source, try again with `make CUDA_VERSION=DETECTED_CUDA_VERSION`,
for example, `make CUDA_VERSION=113`.

The CUDA version for the compile might depend on your conda install, if using conda.
Inspect CUDA version via `conda list | grep cuda`.
To manually override the PyTorch CUDA version please see: https://github.com/TimDettmers/bitsandbytes/blob/main/docs/source/nonpytorchcuda.mdx
CUDA SETUP: WARNING! CUDA runtime files not found in any environmental path.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Checking that the library is importable and CUDA is callable...
Couldn't load the bitsandbytes library, likely due to missing binaries.
Please ensure bitsandbytes is properly installed.

For source installations, compile the binaries with `cmake -DCOMPUTE_BACKEND=cuda -S .`.
See the documentation for more details if needed.

Trying a simple check anyway, but this will likely fail...
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/diagnostics/main.py", line 66, in main
    sanity_check()
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/diagnostics/main.py", line 40, in sanity_check
    adam.step()
  File "/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py", line 385, in wrapper
    out = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/optim/optimizer.py", line 287, in step
    self.update_step(group, p, gindex, pindex)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/optim/optimizer.py", line 496, in update_step
    F.optimizer_update_32bit(
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/functional.py", line 1584, in optimizer_update_32bit
    optim_func = str2optimizer32bit[optimizer_name][0]
NameError: name 'str2optimizer32bit' is not defined
Above we output some debug information.
Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose
WARNING: Please be sure to sanitize sensitive info from the output before posting it.

And standalone:

root@1c1b2c566ff1:/workspace/ilya_png# python
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bitsandbytes
Could not find the bitsandbytes CUDA binary at PosixPath('/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cuda121.so')
Could not load bitsandbytes native library: /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/cextension.py", line 109, in <module>
    lib = get_native_library()
  File "/usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/cextension.py", line 96, in get_native_library
    dll = ct.cdll.LoadLibrary(str(binary_path))
  File "/usr/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.10/dist-packages/bitsandbytes-0.43.2.dev0-py3.10-linux-x86_64.egg/bitsandbytes/libbitsandbytes_cpu.so: cannot open shared object file: No such file or directory

CUDA Setup failed despite CUDA being available. Please run the following command to get more information:

python -m bitsandbytes

Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Python 3.10.12 Ubuntu 22.04 CUDA 12.2 PyTorch 2.2 GPU: NVIDIA A6000 Ada Generation (48Gb)

Installed with:

git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=118 make cuda11x
python setup.py install

Perhaps, I should've picked different CUDA_VERSION

matthewdouglas commented 4 months ago

@NavinKumarMNK Given your GPU compute capability (7.0) you'll want to compile with -DNO_CUBLASLT. I'm curious to know if this works on ppc64le after that adjustment!

@shigabeev It looks like you're trying to run from source too, but the instructions have changed now to use CMake. See here: https://huggingface.co/docs/bitsandbytes/main/en/installation#compile-from-source

You should be using the same CUDA version as PyTorch (in your case, 12.1) or otherwise set BNB_CUDA_VERSION to match the version of the toolkit you're using to build with (i.e. export BNB_CUDA_VERSION=122)

NavinKumarMNK commented 4 months ago 
[ 14%] Building CXX object CMakeFiles/bitsandbytes.dir/csrc/common.cpp.o
In file included from /root/bitsandbytes/include/AAlloc.h:3,
                 from /root/bitsandbytes/include/BinSearch.h:3,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/root/bitsandbytes/include/Portable.h:35:2: error: #error Compiler must support SSE2
   35 | #error Compiler must support SSE2
      |  ^~~~~
In file included from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:56:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   56 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
In file included from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:66,
                 from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/xmmintrin.h:54:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   54 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
In file included from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/xmmintrin.h:79,
                 from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:66,
                 from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/mmintrin.h:52:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   52 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
make[2]: *** [CMakeFiles/bitsandbytes.dir/build.make:76: CMakeFiles/bitsandbytes.dir/csrc/common.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/bitsandbytes.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

i get this error when i build from source

1049451037 commented 4 months ago

Same issue... Sad

13988wzz commented 1 month ago 
[ 14%] Building CXX object CMakeFiles/bitsandbytes.dir/csrc/common.cpp.o
In file included from /root/bitsandbytes/include/AAlloc.h:3,
                 from /root/bitsandbytes/include/BinSearch.h:3,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/root/bitsandbytes/include/Portable.h:35:2: error: #error Compiler must support SSE2
   35 | #error Compiler must support SSE2
      |  ^~~~~
In file included from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:56:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   56 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
In file included from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:66,
                 from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/xmmintrin.h:54:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   54 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
In file included from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/xmmintrin.h:79,
                 from /usr/lib/gcc/powerpc64le-linux-gnu/11/include/emmintrin.h:66,
                 from /root/bitsandbytes/include/SIMD.h:6,
                 from /root/bitsandbytes/include/BinSearch.h:5,
                 from /root/bitsandbytes/csrc/common.h:1,
                 from /root/bitsandbytes/csrc/common.cpp:1:
/usr/lib/gcc/powerpc64le-linux-gnu/11/include/mmintrin.h:52:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
   52 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this error."
      |  ^~~~~
make[2]: *** [CMakeFiles/bitsandbytes.dir/build.make:76: CMakeFiles/bitsandbytes.dir/csrc/common.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/bitsandbytes.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

i get this error when i build from source

vim cmakelists.txt

add_compile_options(-std=c++11) set(CMAKE_CUDA_COMPILER "/usr/local/cuda-11.8/bin/nvcc") set(CMAKE_CUDA_ARCHITECTURES "native")