Installing cuBLAS and cuDNN for CUDA 12?

[chrisspen]

Do you have to install the cuBLAS/cuDNN libraries for CUDA 11, or will it also work with the CUDA 12 versions?

nvidia-smi says my GPU supports CUDA 12.1, so can I get away with using the CUDA 12 libraries?

I also installed CUDA with:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda

I see that's also no provided some Nvidia packages like nvidia-cudnn and libcublas11 but if I try to install them, I get a ton of package conflicts like:

The following packages have unmet dependencies:
 libcuinj64-11.5 : Depends: libnvidia-compute-495 (>= 495) but it is not going to be installed or
                            libnvidia-compute-495-server (>= 495) but it is not installable or
                            libcuda.so.1 (>= 495) or
                            libcuda-11.5-1
 libnvidia-ml-dev : Depends: libnvidia-compute-495 (>= 495) but it is not going to be installed or
                             libnvidia-compute-495-server (>= 495) but it is not installable or
                             libnvidia-ml.so.1 (>= 495)
 nvidia-cuda-dev : Depends: libnvidia-compute-495 (>= 495) but it is not going to be installed or
                            libnvidia-compute-495-server (>= 495) but it is not installable or
                            libcuda.so.1 (>= 495) or
                            libcuda-11.5-1
                   Recommends: libnvcuvid1 but it is not installable
E: Unable to correct problems, you have held broken packages.

Do I already have what I need or do I still need to install cudnn-local-repo-ubuntu2204-8.9.7.29_1.0-1_amd64.deb?

[Purfview]

Do you have to install the cuBLAS/cuDNN libraries for CUDA 11, or will it also work with the CUDA 12 versions?

You've to install the cuBLAS/cuDNN libraries for CUDA 11, libs for CUDA 12 won't work.

You can try to copy manually from -> https://github.com/Purfview/whisper-standalone-win/releases/tag/libs

[Yangqing]

If you have to use cuda 12 and want a quick hacky fix, one way to try is:

ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so.11

since ctranslate2 uses standard cublas apis, this usually won't produce an error. You can keep all other configs as-is.

[swiftraccoon]

ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so.11

can confirm presently working for me on Fedora 39 on python3.9 with cudnn-local-repo-rhel9-9.0.0-1.0-1.x86_64.rpm , nvhpc-24-1-24.1-1.x86_64