search

bitsandbytes-foundation / bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.
https://huggingface.co/docs/bitsandbytes/main/en/index
MIT License
6.11k stars 611 forks source link

ModuleNotFoundError: No module named 'triton.language' #561

Closed chauvinSimon closed 11 months ago

chauvinSimon commented 1 year ago

Hi, I am trying to install bitsandbytes.

in short:

Software requirements

I think I respect the software requirements:

some of my python packages:

(draft) simon-chauvin@PF48QN5A:~/drafts/bitsandbytes$ conda list |grep cuda
cuda-cudart               11.7.99                       0    nvidia
cuda-cupti                11.7.101                      0    nvidia
cuda-libraries            11.7.1                        0    nvidia
cuda-nvrtc                11.7.99                       0    nvidia
cuda-nvtx                 11.7.91                       0    nvidia
cuda-runtime              11.7.1                        0    nvidia
cudatoolkit               11.3.1               h2bc3f7f_2    anaconda
nvidia-cuda-cupti-cu11    11.7.101                 pypi_0    pypi
nvidia-cuda-nvrtc-cu11    11.7.99                  pypi_0    pypi
nvidia-cuda-runtime-cu11  11.7.99                  pypi_0    pypi
pytorch-cuda              11.7                 h778d358_5    pytorch
pytorch-mutex             1.0                        cuda    pytorch

nvcc version:

(draft) simon-chauvin@PF48QN5A:~/drafts/bitsandbytes$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Hardware requirements

I am not sure if I respect the Hardware requirements:

I have a nvidia RTX A2000 with 8Gb. Is it enough?

nvidia-smi:

(draft) simon-chauvin@PF48QN5A:~/drafts/bitsandbytes$ nvidia-smi
Tue Jul  4 12:21:42 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A200...  Off  | 00000000:01:00.0  On |                  N/A |
| N/A   39C    P8     8W /  60W |    106MiB /  8192MiB |     37%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

python -m bitsandbytes

I have run pip install bitsandbytes then 

(draft) simon-chauvin@PF48QN5A:~/drafts/bitsandbytes$ python -m bitsandbytes 

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/simon-chauvin/drafts/bitsandbytes/bitsandbytes/libbitsandbytes_cpu.so
False
/home/simon-chauvin/drafts/bitsandbytes/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
CUDA SETUP: Required library version not found: libbitsandbytes_cpu.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

following https://github.com/TimDettmers/bitsandbytes/issues/156#issuecomment-1474056975 I run

cd ~/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes
cp libbitsandbytes_cuda117.so libbitsandbytes_cpu.so

then 

(draft) simon-chauvin@PF48QN5A:~/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes$ python -m bitsandbytes 

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
CUDA SETUP: Loading binary /home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
Traceback (most recent call last):
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes/__init__.py", line 16, in <module>
    from .nn import modules
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes/nn/__init__.py", line 6, in <module>
    from .triton_based_modules import SwitchBackLinear, SwitchBackLinearGlobal, SwitchBackLinearVectorwise, StandardLinear
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes/nn/triton_based_modules.py", line 8, in <module>
    from bitsandbytes.triton.dequantize_rowwise import dequantize_rowwise
  File "/home/simon-chauvin/anaconda3/envs/draft/lib/python3.8/site-packages/bitsandbytes/triton/dequantize_rowwise.py", line 11, in <module>
    import triton.language as tl
ModuleNotFoundError: No module named 'triton.language'

strange, since triton == 2.0.0 is installed

Daryl149 commented 1 year ago

Yep, had the same issue here. It's because bitsandbytes can't find triton anymore after your cp libbitsandbytes_cuda117.so libbitsandbytes_cpu.so command Deleting triton manually from my site-packages and reinstalling with pip install triton fixes this error.

TimDettmers commented 1 year ago

It is interesting that a triton install solved this issue. The newest version of trition seems to be 2.0.0.post1, so maybe this version has major changes and is the requirement for bitsandbytes? I need to check this in more detail.

Daryl149 commented 1 year ago

Interestingly, as soon as you run the manual cp libbitsandbytes_cuda117.so libbitsandbytes_cpu.so, it can still find import triton, but not import triton.language, until you reinstall triton. Since it seems like the manual cp breaks it, if we don't need to run the cp to make our cuda detectable, then maybe this bug is solved automatically?

Titus-von-Koeller commented 11 months ago

Tim and I just discovered what this issue was about, @Daryl149:

When copying the .so you were likely in the bitsandbytes/bitsandbytes/ sub-directory, which has a triton sub-directory.

Therefore, this must have been a namespace collision, importing from the sub-directory instead of the installed package. Likely, when you reinstalled triton you also reran your commands from a different directory and therefore it appeared as if that was what solved the issue.

Closing this issue. Let me know if you disagree.

swumagic commented 10 months ago

OR you are Linux distribution (Ubuntu, MacOS, etc.)system ,AND CUDA Version: 11.X.

Bitsandbytes can support ubuntu.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/TimDettmers/bitsandbytes/releases/download/0.41.0/bitsandbytes-0.41.0-py3-none-any.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）