nvidia-smi is a useful tool to query information about the GPUs present in a system.
In general, this should not be relevant to most users, but may be useful in some situations.
nvidia -smi topo -m is likely to be one of the useful sucommands.
It will be worth noting that:
Many commands require root access, so will not be available to users
Only GPUs available to the job will be visible (i.e. If you request 2 GPUs, only 2 devices should be listed assuming cgroups is used to control access)
Device enumeration in nvidia-smi may not match device enumeration within CUDA. This is controlled by a (kernel module?) setting, and the CUDA_VISIBLE_DEVICES variable can also influence order if set by a user.
nvidia-smiis a useful tool to query information about the GPUs present in a system.In general, this should not be relevant to most users, but may be useful in some situations.
nvidia -smi topo -mis likely to be one of the useful sucommands.It will be worth noting that:
nvidia-smimay not match device enumeration within CUDA. This is controlled by a (kernel module?) setting, and theCUDA_VISIBLE_DEVICESvariable can also influence order if set by a user.