search

bitsandbytes-foundation / bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.
https://huggingface.co/docs/bitsandbytes/main/en/index
MIT License
6.05k stars 609 forks source link

bitsandbytes broken with regards to cuda with name error 'str2optimizer32bit' and name 'str2optimizer8bit_blockwise' is not defined #1260

Open artsera opened 2 months ago

artsera commented 2 months ago

System Info

Intel Core i7-9700KF Processor 3.6GHz with Nvidia GTX 1070 8Gb Debian 12 Linux Built bitsandbytes from source code with GNU gcc 11.3.0 in accordance with instructions from: https://github.com/TimDettmers/bitsandbytes/blob/main/docs/source/installation.mdx

Reproduction

Here is a colab reference for code that produces the error under python 3.10:

https://colab.research.google.com/drive/1AErkPgDderPW0dgE230OOjEysd0QV1sR?usp=sharing#scrollTo=U77JPibZ4QuK

Expected behavior

I would expect the code to work or give me a useful error message that allowed me to make whatever adjustments would be necessary to avoid the error situation.

Titus-von-Koeller commented 2 months ago

Dear @artsera,

Please give the full error trace you're seeing and if it didn't come from running python -m bitsandbytes then run that and also post the output of that here as well.

The error you're describing can pop up if the instance where you're running things doesn't have a functional CUDA installation. Therefore, please also paste the output of nvidia-smi.

In a collab notebook you can run shell commands by prepending a !, so you'd have to run

!nvidia-smi
!python -m bitsandbytes