Whether to support multi-machine training ？

kakaobrain / torchgpipe

A GPipe implementation in PyTorch

https://torchgpipe.readthedocs.io/

BSD 3-Clause "New" or "Revised" License

802 stars 98 forks source link

Closed yangpc615 closed 4 years ago

yangpc615 commented 4 years ago

Now， I have two machine with two GPUS, and I want to use them to train a large model, Can it be reached by torchgpipe ?

sublee commented 4 years ago

For now, we don't have a plan to support multi-node training yet.

eric-haibin-lin commented 3 years ago

Is this still the case?

sublee commented 3 years ago

@eric-haibin-lin torchgpipe still does not support pipeline parallelism across multiple hosts.

amsword commented 2 years ago

is there any alternative which supports multi-node pipelining?