Build Failing Cuda 12.1

[sshleifer]

Build CMD

CUDA_HOME=/usr/local/cuda LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda CUDA_VERSION=12.1 make cuda12x

• also tried export BNB_CUDA_VERSION=121 before

Other packages

 torch.version.cuda
'12.1'

pip freeze | grep cuda

cuda-python @ file:///rapids/cuda_python-12.1.0rc5%2B1.gc7fd38c.dirty-cp38-cp38-linux_x86_64.whl
cupy-cuda12x @ file:///rapids/cupy_cuda12x-12.0.0b3-cp38-cp38-linux_x86_64.whl
dask-cuda @ file:///rapids/dask_cuda-23.2.0-py3-none-any.whl
nvidia-dali-cuda110==1.23.0
(cai-env) ➜  ~ pip freeze | grep -E 'cuda|torch|nvidia'
cuda-python @ file:///rapids/cuda_python-12.1.0rc5%2B1.gc7fd38c.dirty-cp38-cp38-linux_x86_64.whl
cupy-cuda12x @ file:///rapids/cupy_cuda12x-12.0.0b3-cp38-cp38-linux_x86_64.whl
dask-cuda @ file:///rapids/dask_cuda-23.2.0-py3-none-any.whl
nvidia-dali-cuda110==1.23.0
nvidia-pyindex==1.0.9
onnx @ file:///opt/pytorch/pytorch/third_party/onnx
pycocotools @ git+https://github.com/nvidia/cocoapi.git@8b8fd68576675c3ee77402e61672d65a7d826ddf#subdirectory=PythonAPI
pytorch-quantization==2.1.2
torch==2.0.0a0+1767026
torch-tb-profiler==0.3.1
torch-tensorrt @ file:///opt/pytorch/torch_tensorrt/py/dist/torch_tensorrt-1.4.0.dev0-cp38-cp38-linux_x86_64.whl
torchtext @ git+https://github.com/pytorch/text@fae8e8cabf7adcbbc2f09c0520216288fd53f33b
torchtyping==0.1.4
torchvision @ file:///opt/pytorch/vision

(cai-env) ➜  bitsandbytes git:(main) ldconfig -p | grep libcudart.so
        libcudart.so.12 (libc6,x86-64) => /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.12
        libcudart.so.11.0 (libc6,x86-64) => /usr/local/cuda-11/targets/x86_64-linux/lib/libcudart.so.11.0
        libcudart.so (libc6,x86-64) => /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so
        libcudart.so (libc6,x86-64) => /usr/local/cuda-11/targets/x86_64-linux/lib/libcudart.so

python -m bitsandbytes

===================================BUG REPORT===================================
/home/sam/bitsandbytes/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

  warn(msg)
================================================================================
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib')}
/home/sam/bitsandbytes/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda-11/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda-11/lib64/libcudart.so')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
  warn(msg)
/home/sam/bitsandbytes/bitsandbytes/cuda_setup/main.py:166: UserWarning: /usr/local/lib/python3.8/dist-packages/torch/lib:/usr/local/lib/python3.8/dist-packages/torch_tensorrt/lib:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-11/lib64::/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
  warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('1'), PosixPath('/usr/share/modules/$MODULE_VERSION/modulefiles')}
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/share/modules/$MODULE_VERSION/modulefiles')}
The following directories listed in your path were found to be non-existent: {PosixPath('/home/sam/.oh-my-zsh/functions'), PosixPath('/usr/share/zsh/vendor-functions'), PosixPath('/home/sam/.oh-my-zsh/completions')}
The following directories listed in your path were found to be non-existent: {PosixPath('gs'), PosixPath('//shleifer-ckpts/ckpts')}
The following directories listed in your path were found to be non-existent: {PosixPath('/opus.spm32k-spm32k.vocab.yml')}
The following directories listed in your path were found to be non-existent: {PosixPath('/home/sam/ParlAI')}
The following directories listed in your path were found to be non-existent: {PosixPath('blender/blender_90M/model'), PosixPath('zoo')}
The following directories listed in your path were found to be non-existent: {PosixPath('zoo'), PosixPath('blender/blender_3B/model')}
The following directories listed in your path were found to be non-existent: {PosixPath('data/models/blender/blender_3B/model')}
The following directories listed in your path were found to be non-existent: {PosixPath('data/models/blender/blender_90M/model')}
The following directories listed in your path were found to be non-existent: {PosixPath('logs/bb12/v2')}
The following directories listed in your path were found to be non-existent: {PosixPath('//blob/checkpoints'), PosixPath('az')}
The following directories listed in your path were found to be non-existent: {PosixPath('//character-ai-eu-frankfurt-1/checkpoints/sam'), PosixPath('s3')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=121, Highest Compute Capability: 9.0.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Required library version not found: libbitsandbytes_cuda121.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

[az077xtz]

currently it only supports 12.0

[swumagic]

Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.