Open hiberfil opened 2 months ago
Hi @hiberfil,
Thank you for choosing our framework :)
So far we do not have plans about adding native distributed data parallels support. However, it will be great to have, therefore any contributions are very welcome.
Also, thank you for providing a simple workaround script, it will be definitely useful for others!
Hi, I was wondering if there were any efforts on great.py natively supporting Distributed Data Parallels? Currently I am doing a workaround by editing my own trainer file and saving it via torch save.
Below is how I invoke it.
torchrun --nproc_per_node=8 ddptest.py
Again thank you so much for this awesome framework.