Did byteps using NCCL all-reduce with co-locate mode?

bytedance / byteps

A high performance and generic framework for distributed DNN training

Other

3.62k stars 487 forks source link

Did byteps using NCCL all-reduce with co-locate mode? #374

Open Ruinhuang opened 3 years ago

Ruinhuang commented 3 years ago

if i use co-locate mode (with no additional CPU servers, for each physical machine, launch a worker and a server process), Did byteps using NCCL all-reduce?

eric-haibin-lin commented 3 years ago

BytePS uses NCCL for intra-node communication and ps-lite for inter-node communication, including colocate mode.