search

sail-sg / MDT

Masked Diffusion Transformer is the SOTA for image synthesis. (ICCV 2023)
Apache License 2.0
500 stars 35 forks source link

Training Time #42

Open mapengsen opened 4 months ago

mapengsen commented 4 months ago

Dear Authors,

when i use one gpu to train the model : python -m torch.distributed.launch --nproc_per_node=1 scripts/image_train.py

and use 4 gpu to train the model: python -m torch.distributed.launch --nproc_per_node=4 scripts/image_train.py

The number of training steps is the same under the same training time (the final modelXXX.pt obtained by 4 GPUs will not be more/faster than that obtained by 1 GPU). Is this normal?

mapengsen commented 4 months ago

if i need set "--world_size 4"?