qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.98k stars 457 forks source link

No CUDA_ENV / conda-froce cudatoolkit-dev freezes #245

Closed nathanleclaire closed 1 year ago

nathanleclaire commented 1 year ago

I had an unfortunate time installing the cuda toolkit on WSL to use this

Unrelated to repo directly but, conda install -c conda-forge cudatoolkit-dev just hung forever and couldn't solve

nvidia-cuda-toolkit was out of date to 11.7 so I fixed it by following these steps:

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

Just documenting here so it's hopefully helpful to someone down the line.