CausalVAE training does not have DDP support

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

MIT License

11.23k stars 999 forks source link

CausalVAE training does not have DDP support #359

Open ahmad-573 opened 1 month ago

ahmad-573 commented 1 month ago

I am trying to train the CausalVAE on my own dataset on 4 gpus but all memory is just used by device 0. Is distributed processing not incorporated into the training code?

qqingzheng commented 1 month ago

set CUDA_VISABLE_DEVICES in training script.