search

NVIDIA / nccl

Optimized primitives for collective multi-GPU communication
Other
3.28k stars 831 forks source link

Does ncclBroadcast call return at same time on different ranks？ #1294

Open Eiji911 opened 6 months ago

Eiji911 commented 6 months ago

image

for rank0，when ncclBroadcast call returned，does rank3 has received data from rank2？