search

bitsandbytes-foundation / bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.
https://huggingface.co/docs/bitsandbytes/main/en/index
MIT License
6.26k stars 626 forks source link

str2optimizer32bit - 8-bit optimizers and GPU quantization are unavailable #789

Closed jorgeceballosgarcia closed 10 months ago

jorgeceballosgarcia commented 1 year ago

Hi,

Output "python -m bitsandbytes"

/home/opc/sd/stable-diffusion-webui/venv/lib64/python3.11/site-packages/bitsandbytes/cextension.py:127: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++ DEBUG INFORMATION +++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

++++++++++ POTENTIALLY LIBRARY-PATH-LIKE ENV VARS ++++++++++ 'LD_LIBRARY_PATH': '/usr/local/cuda-12.0/targets/x86_64-linux/lib/:/usr/local/cuda-12.0/targets/x86_64-linux/lib/:' 'MODULES_CMD': '/usr/share/Modules/libexec/modulecmd.tcl' 'CUDA_HOME': '/usr/local/cuda' 'MODULEPATH': '/etc/scl/modulefiles:/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles' 'MODULEPATH_modshare': '/usr/share/Modules/modulefiles:2:/etc/modulefiles:2:/usr/share/modulefiles:2' 'MODULESHOME': '/usr/share/Modules' 'BASH_FUNC_which%%': '() { ( alias; eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot $@ }' 'BASH_FUNC__module_raw%%': '() { unset _mlshdbg; if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then case "$-" in vx) set +vx; _mlshdbg='vx' ;; v) set +v; _mlshdbg='v' ;; x) set +x; _mlshdbg='x' ;; ) _mlshdbg='' ;; esac; fi; unset _mlre _mlIFS; if [ -n "${IFS+x}" ]; then _mlIFS=$IFS; fi; IFS=' '; for _mlv in ${MODULES_RUN_QUARANTINE:-}; do if [ "${_mlv}" = "${mlv##*[!A-Za-z0-9]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then if [ -n "eval 'echo ${'$_mlv'+x}'" ]; then _mlre="${_mlre:-}${_mlv}_modquar='eval 'echo ${'$_mlv'}'' "; fi; _mlrv="MODULESRUNENV${_mlv}"; _mlre="${_mlre:-}${_mlv}='eval 'echo ${'$_mlrv':-}'' "; fi; done; if [ -n "${_mlre:-}" ]; then eval eval ${_mlre} /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash '"$@"'; else eval /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"; fi; _mlstatus=$?; if [ -n "${_mlIFS+x}" ]; then IFS=$_mlIFS; else unset IFS; fi; unset _mlre _mlv _mlrv _mlIFS; if [ -n "${_mlshdbg:-}" ]; then set -$_mlshdbg; fi; unset _mlshdbg; return $_mlstatus }' 'BASH_FUNC_switchml%%': '() { typeset swfound=1; if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then typeset swname='main'; if [ -e /usr/share/Modules/libexec/modulecmd.tcl ]; then typeset swfound=0; unset MODULES_USE_COMPAT_VERSION; fi; else typeset swname='compatibility'; if [ -e /usr/share/Modules/libexec/modulecmd-compat ]; then typeset swfound=0; MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION; fi; fi; if [ $swfound -eq 0 ]; then echo "Switching to Modules $swname version"; source /usr/share/Modules/init/bash; else echo "Cannot switch to Modules $swname version, command not found"; return 1; fi }' 'BASH_FUNC_scl%%': '() { if [ "$1" = "load" -o "$1" = "unload" ]; then eval "module $@"; else /usr/bin/scl "$@"; fi }' ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

WARNING: Please be sure to sanitize sensible info from any such env vars!

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++ COMPILED_WITH_CUDA = False COMPUTE_CAPABILITIES_PER_GPU = ['8.6', '8.6'] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:

  • library is importable
  • CUDA function is callable

name 'str2optimizer32bit' is not defined

Above we output some debug information. Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose ...

Output nvidia-smi

Mon Sep 25 15:43:07 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.125.06 Driver Version: 525.125.06 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA A10 Off | 00000000:00:04.0 Off | 0 | | 0% 41C P0 54W / 150W | 3562MiB / 23028MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 1 NVIDIA A10 Off | 00000000:00:05.0 Off | 0 | | 0% 27C P8 8W / 150W | 2MiB / 23028MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 91602 C python3 3560MiB | +-----------------------------------------------------------------------------+

Output "ls" from site-package

total 169140 -rwxr-xr-x. 1 opc opc 14504328 Sep 25 15:37 libbitsandbytes_cuda120.so -rw-rw-r--. 1 opc opc 322 Sep 25 15:38 debug_cli.py -rw-rw-r--. 1 opc opc 6892 Sep 25 15:38 cextension.py -rw-rw-r--. 1 opc opc 2152 Sep 25 15:38 main.py -rw-rw-r--. 1 opc opc 644 Sep 25 15:38 init.py -rwxrwxr-x. 1 opc opc 41608 Sep 25 15:38 libbitsandbytes_cpu.so -rw-rw-r--. 1 opc opc 60059 Sep 25 15:38 functional.py -rwxrwxr-x. 1 opc opc 10418376 Sep 25 15:38 libbitsandbytes_cuda102_nocublaslt.so -rwxrwxr-x. 1 opc opc 2565864 Sep 25 15:38 libbitsandbytes_cuda110.so -rwxrwxr-x. 1 opc opc 10285456 Sep 25 15:38 libbitsandbytes_cuda110_nocublaslt.so -rwxrwxr-x. 1 opc opc 3917544 Sep 25 15:38 libbitsandbytes_cuda111.so -rwxrwxr-x. 1 opc opc 12173712 Sep 25 15:38 libbitsandbytes_cuda111_nocublaslt.so -rwxrwxr-x. 1 opc opc 4089480 Sep 25 15:38 libbitsandbytes_cuda112.so -rwxrwxr-x. 1 opc opc 12669232 Sep 25 15:38 libbitsandbytes_cuda112_nocublaslt.so -rwxrwxr-x. 1 opc opc 4134536 Sep 25 15:38 libbitsandbytes_cuda113.so -rwxrwxr-x. 1 opc opc 12869936 Sep 25 15:38 libbitsandbytes_cuda113_nocublaslt.so -rwxrwxr-x. 1 opc opc 4163208 Sep 25 15:38 libbitsandbytes_cuda114.so -rwxrwxr-x. 1 opc opc 12939568 Sep 25 15:38 libbitsandbytes_cuda114_nocublaslt.so -rwxrwxr-x. 1 opc opc 4167640 Sep 25 15:38 libbitsandbytes_cuda115.so -rwxrwxr-x. 1 opc opc 12886656 Sep 25 15:38 libbitsandbytes_cuda115_nocublaslt.so -rwxrwxr-x. 1 opc opc 4167640 Sep 25 15:38 libbitsandbytes_cuda116.so -rwxrwxr-x. 1 opc opc 12898944 Sep 25 15:38 libbitsandbytes_cuda116_nocublaslt.so -rwxrwxr-x. 1 opc opc 4172040 Sep 25 15:38 libbitsandbytes_cuda117.so -rwxrwxr-x. 1 opc opc 12903352 Sep 25 15:38 libbitsandbytes_cuda117_nocublaslt.so -rwxrwxr-x. 1 opc opc 4160056 Sep 25 15:38 libbitsandbytes_cuda118.so -rw-rw-r--. 1 opc opc 686 Sep 25 15:38 utils.py -rwxrwxr-x. 1 opc opc 12895464 Sep 25 15:38 libbitsandbytes_cuda118_nocublaslt.so drwxrwxr-x. 3 opc opc 65 Sep 25 15:38 autograd drwxrwxr-x. 3 opc opc 94 Sep 25 15:38 cuda_setup drwxrwxr-x. 3 opc opc 62 Sep 25 15:38 nn drwxrwxr-x. 3 opc opc 175 Sep 25 15:38 optim drwxrwxr-x. 2 opc opc 4096 Sep 25 15:38 pycache drwxrwxr-x. 7 opc opc 4096 Sep 25 15:38 . drwxr-xr-x. 387 opc opc 16384 Sep 25 15:38 ..

Ideas to solve?

Regards

swumagic commented 12 months ago

Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

github-actions[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.