Hello, my server has a CUDA version of 11.1 but I am struggling to use the latest bitsandbytes package since it support CUDA from 11.7 ~ 12.5, where mine is outside the range.
Thus, I tried to manually downgrade the version step by step so that I could find the runnable version with my CUDA 11.1 near bitsandbytes==0.40.0. However, the following error occurs:
RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback):
CUDA Setup failed despite GPU being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
, where I found the solution was just to upgrade the package version.
After upgrading via pip install -U bitsandbytes, however,
Hello, my server has a CUDA version of 11.1 but I am struggling to use the latest bitsandbytes package since it support CUDA from 11.7 ~ 12.5, where mine is outside the range.
Thus, I tried to manually downgrade the version step by step so that I could find the runnable version with my CUDA 11.1 near
bitsandbytes==0.40.0. However, the following error occurs:, where I found the solution was just to upgrade the package version.
After upgrading via
pip install -U bitsandbytes, however,error occurs. The thing is, of course there is no .so file for cuda111 afterwards.
Anyone knows the proper bitsandbytes version for 4-bit quantization supporting CUDA 11.1?
Best, Junyeong Ahn