NVIDIA / nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs
Apache License 2.0
1.88k stars 214 forks source link

Increase priority of ld.so.conf.d config file #527

Closed elezar closed 2 weeks ago

elezar commented 3 weeks ago

This change ensures that the created /etc/ld.so.conf.d file has a higher priority to ensure that the injected libraries take precendence.

For example:

ls -l /etc/ld.so.conf.d/
total 48
-rw-r--r--. 1 root root  27 Mar 16 03:23 00-cuda-compat.conf
-rw-r--r--. 1 root root  75 Jun  4 10:37 00-nvcr-XXXXXX.conf
-rw-r--r--. 1 root root  42 Aug 16  2023 000_cuda.conf
-rw-r--r--. 1 root root  45 Aug 16  2023 988_cuda-12.conf
-rw-r--r--. 1 root root 103 Mar  4  2022 aarch64-linux-gnu.conf
-rw-r--r--. 1 root root  22 Mar  9 00:31 cuda.conf
-rw-r--r--. 1 root root  96 Apr 29 07:01 deepstream.conf
-rw-r--r--. 1 root root  47 Aug 15  2023 gds-12-2.conf
-rw-r--r--. 1 root root 132 Mar  9 00:31 hpcx.conf
-rw-r--r--. 1 root root  44 Dec 16  2020 libc.conf
-rw-r--r--. 1 root root  70 Mar  9 00:25 nvidia-tegra.conf
-rw-r--r--. 1 root root  46 Mar  9 00:27 nvidia.conf
lrwxrwxrwx. 1 root root  43 Apr 29 06:58 vpi3.conf -> /opt/nvidia/vpi3/etc/ld.so.conf.d/vpi3.conf

Note that this may require special attention for the CUDA compat libraries.

 for f in $(ls /etc/ld.so.conf.d/*.conf); do echo "## Contents of $f:"; cat $f; done
## Contents of /etc/ld.so.conf.d/00-cuda-compat.conf:
/usr/local/cuda/compat/lib
## Contents of /etc/ld.so.conf.d/00-nvcr-XXXXXX.conf:
/usr/lib64/nvidia
/usr/lib64
/usr/lib64/gstreamer-1.0
/usr/lib64/tegra-egl
## Contents of /etc/ld.so.conf.d/000_cuda.conf:
/usr/local/cuda/targets/aarch64-linux/lib
## Contents of /etc/ld.so.conf.d/988_cuda-12.conf:
/usr/local/cuda-12/targets/aarch64-linux/lib
## Contents of /etc/ld.so.conf.d/aarch64-linux-gnu.conf:
# Multiarch support
/usr/local/lib/aarch64-linux-gnu
/lib/aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu
## Contents of /etc/ld.so.conf.d/cuda.conf:
/usr/local/cuda/lib64
## Contents of /etc/ld.so.conf.d/deepstream.conf:
/opt/nvidia/deepstream/deepstream-7.0/lib
/opt/nvidia/deepstream/deepstream-7.0/lib/gst-plugins
## Contents of /etc/ld.so.conf.d/gds-12-2.conf:
/usr/local/cuda-12.2/targets/aarch64-linux/lib
## Contents of /etc/ld.so.conf.d/hpcx.conf:
/opt/hpcx/hcoll/lib
/opt/hpcx/nccl_rdma_sharp_plugin/lib
/opt/hpcx/ompi/lib
/opt/hpcx/sharp/lib
/opt/hpcx/ucc/lib
/opt/hpcx/ucx/lib
## Contents of /etc/ld.so.conf.d/libc.conf:
# libc default configuration
/usr/local/lib
## Contents of /etc/ld.so.conf.d/nvidia-tegra.conf:
/usr/lib/aarch64-linux-gnu/tegra
/usr/lib/aarch64-linux-gnu/tegra-egl
## Contents of /etc/ld.so.conf.d/nvidia.conf:
/usr/local/nvidia/lib
/usr/local/nvidia/lib64
## Contents of /etc/ld.so.conf.d/vpi3.conf:
/opt/nvidia/vpi3/lib/aarch64-linux-gnu