Open jabberjabberjabber opened 6 months ago
@jabberjabberjabber Oh no more bitsandbytes issues :((
@danielhanchen I understand that we need to have same cudaversion for both pytorch and bitsandbytes, but it seems that bitsandbytes doesn't support cuda12 properly. Can you please check, this problem is very old and I have raised this issue earlier as well while working with LLamaFactory.
@sahilqure Actually was gonna ask did you install bitsandbytes via Conda or pip? If you did pip - maybe first try uninstalling it, then install it with Conda / Mamba
@sahilqure Actually was gonna ask did you install bitsandbytes via Conda or pip? If you did pip - maybe first try uninstalling it, then install it with Conda / Mamba
I used this guide to install: https://github.com/unslothai/unsloth/issues/73
Do you know how to install bitsandbytes via conda? This is what happened when I tried:
(unsloth_env) user@t7610:~$ pip uninstall bitsandbytes
Found existing installation: bitsandbytes 0.41.1
Uninstalling bitsandbytes-0.41.1:
Would remove:
/home/user/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/bitsandbytes-0.41.1.dist-info/*
/home/user/miniconda3/envs/unsloth_env/lib/python3.10/site-packages/bitsandbytes/*
Proceed (Y/n)? y
Successfully uninstalled bitsandbytes-0.41.1
(unsloth_env) user@t7610:~$ conda install bitsandbytes
Retrieving notices: ...working... done
Channels:
- defaults
- nvidia
- pytorch
- xformers
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- bitsandbytes
Current channels:
- defaults
- https://conda.anaconda.org/nvidia/linux-64
- https://conda.anaconda.org/pytorch/linux-64
- https://conda.anaconda.org/pytorch/noarch
- https://conda.anaconda.org/xformers/linux-64
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
OK, I got it working by reinstalling using CUDA 11.8 instead of 12.1
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install bitsandbytes
pip install ipython
Oh great!! Glad you managed to solve it! Sorry on the issue again!
Install problem?