NVIDIA / nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs
Apache License 2.0
2.28k stars 245 forks source link

Detect minimum required GLIBC version for injected drivers #543

Open elezar opened 3 months ago

elezar commented 3 months ago

For newer Tegra-based systems certain driver files have specific GLIBC requirements. For example:

root@11320b0ce1f0:/# LD_PRELOAD=/usr/lib/aarch64-linux-gnu/tegra/libnvidia-ml.so.1 nvidia-smi
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvrm_surface.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvrm_sync.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvos.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvos.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvsciipc.so)
nvidia-smi: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/aarch64-linux-gnu/tegra/libnvsciipc.so)

This means that certain container images are not compatible with this driver.

This information is not directly available to the user and would manifest in the forms of errors such as:

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

We could improve the user experience around this by:

See also: