Open sammcj opened 1 month ago
FYI I did a custom build with CUDA 12.5.1 as the base image and it had the same issue.
same issue 1xA6000 + 1x4090
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0
CUDA Version: 12.2
I fixed the following:
sudo apt-get --purge remove "*cuda*"
sudo apt-get autoremove
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-12-4
Describe the bug
When trying to run the latest official cuda-86 image (ghcr.io/ericlbuehler/mistral.rs:cuda-86-latest) mistralrs server fails to load with an error stating it can't find the cuda libs in the LD_LIBRARY_PATH:
Overriding LD_LIBRARY_PATH to include the path to libcuda.so inside the container (
/usr/local/cuda-12.4/compat
) then reveals that the problem is a CUDA version mismatch:LD_LIBRARY_PATH= /usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-12.4/compat
docker inspect shows the env to be:
Latest commit
Host system: