Right now, if there is an NVLink between two GPUs, we assign the same distance in the matrix.
We can query the actual NVLink bandwidth to scale that value.
Furthermore, instead of using magic numbers for common ancestors, we could enumerate the actual system topology and assign values that way.
Right now, if there is an NVLink between two GPUs, we assign the same distance in the matrix. We can query the actual NVLink bandwidth to scale that value.
Furthermore, instead of using magic numbers for common ancestors, we could enumerate the actual system topology and assign values that way.