EESSI / software-layer

Software layer of the EESSI project
https://eessi.github.io/docs/software_layer
GNU General Public License v2.0
23 stars 46 forks source link

Allow Nvidia driver script to set LD_PRELOAD #754

Open ocaisa opened 4 days ago

eessi-bot[bot] commented 4 days ago

Instance eessi-bot-mc-aws is configured to build for:

eessi-build-deploy-bot-deucalion[bot] commented 4 days ago

Instance boegel-bot-deucalion is configured to build for:

eessi-bot[bot] commented 4 days ago

Instance eessi-bot-mc-azure is configured to build for:

ocaisa commented 4 days ago

Example output:

[rocky@ip-172-31-27-81 software-layer]$  ./scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh --ld-preload --no-download
Found NVIDIA GPU driver version 545.23.08
Found host CUDA version 12.3
Using default list of libraries
Matched 48 CUDA Libraries

When attempting to use LD_PRELOAD we exclude anything related to graphics
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libGL.so.1.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libGL.so.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libGLX_nvidia.so.0.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libGLX.so.0.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libGLX.so.
libwayland-server.so.0 is NOT in the provided  preload list, filtering /lib64/libnvidia-egl-wayland.so.1.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libnvidia-fbc.so.1.
libXext.so.6 is NOT in the provided  preload list, filtering /lib64/libnvidia-fbc.so.
libXNVCtrl.so.0 is NOT in the provided  preload list, filtering /lib64/libnvidia-gtk3.so.545.23.08.

The recommended way to use LD_PRELOAD is to only use it when you need to:

export EESSI_GPU_LD_PRELOAD="/lib64/libcuda.so.1:/lib64/libcuda.so:/lib64/libcudadebugger.so.1:/lib64/libnvcuvid.so.1:/lib64/libnvcuvid.so:/lib64/libnvidia-cfg.so.1:/lib64/libnvidia-cfg.so:/lib64/libnvidia-eglcore.so.545.23.08:/lib64/libnvidia-encode.so.1:/lib64/libnvidia-encode.so:/lib64/libnvidia-glcore.so.545.23.08:/lib64/libnvidia-glsi.so.545.23.08:/lib64/libnvidia-glvkspirv.so.545.23.08:/lib64/libnvidia-gpucomp.so.545.23.08:/lib64/libnvidia-ml.so.1:/lib64/libnvidia-ml.so:/lib64/libnvidia-nvvm.so.4:/lib64/libnvidia-nvvm.so:/lib64/libnvidia-opencl.so.1:/lib64/libnvidia-opticalflow.so.1:/lib64/libnvidia-ptxjitcompiler.so.1:/lib64/libnvidia-ptxjitcompiler.so:/lib64/libnvidia-rtcore.so.545.23.08:/lib64/libnvidia-tls.so.545.23.08:/lib64/libnvoptix.so.1:/lib64/libOpenCL.so.1"
export EESSI_OVERRIDE_GPU_CHECK="1"

Then you can set LD_PRELOAD only when you want to run a GPU application, e.g.,
    LD_PRELOAD="$EESSI_GPU_LD_PRELOAD" device_query