search

NVIDIA / nccl

Optimized primitives for collective multi-GPU communication
Other
3.28k stars 831 forks source link

Understanding LL, LL128, and Simple Protocols #1315

Open arkhadem opened 5 months ago

arkhadem commented 5 months ago

Hi Everyone,

I am new to the NCCL and I want to learn more about how different protocols, i.e., LL, LL128, and Simple work.

However, the NCCL code is very hard to understand, and I could not find any good resources for these protocols.

I would appreciate it if anyone referred me to any document explaining how GPUs interact in each of these mentioned protocols.

Best, Alireza