search

DachengLi1 / LongChat

Official repository for LongChat and LongEval
Apache License 2.0
504 stars 29 forks source link

Multi-node training? #11

Closed XueFuzhao closed 1 year ago

XueFuzhao commented 1 year ago

Hi, Thanks for the great work! Are there any support for multi-node training?

DachengLi1 commented 1 year ago

@XueFuzhao Thanks for trying out! Yes, we are compatible with native PyTorch multi-node training. For instance, use --nproc_per_node=8 --nnodes nodes --node_rank=rank --master_addr master.