Open AnjirwalaAnuj opened 3 months ago
Note that nvswitches and nvlinks may not automatically be mounted inside the container. See https://github.com/NVIDIA/dcgm-exporter/issues/316#issuecomment-2087369233
Thank you for your reply. I tried mounting nvswitches and nvlinks devices to the dcgm-exporter container by following https://github.com/NVIDIA/dcgm-exporter/issues/169#issuecomment-1604771610. However, I don't see nvidia-nvswitch* and nvidia-nvlink device files under /dev directory on GH200 nodes. I also tried running dcgm-exporter binary (built from source) and it still couldn't discover the nvswitches and nvlinks.
Ask your question
I am running dcgm-exporter within docker container on a GH200 gpu node. However, the dcgm-exporter is not able to discover NvSwitch and NvLink devices and as a result doesn't export any NvLink metrics. I'm using
nvcr.io/nvidia/k8s/dcgm-exporter:3.3.6-3.4.2-ubuntu22.04
dcgm-exporter image which is latest. Does this latest version of dcgm-exporter support NvLink metrics on GH200 gpu node? If yes, is there any extra configuration required to get NvLink metrics?Below is the dcgm-exporter container logs: