3D DIT pre-training weights？

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

MIT License

11.54k stars 1.03k forks source link

3D DIT pre-training weights？ #363

Open RanXu2000 opened 3 months ago

RanXu2000 commented 3 months ago

Great work, I was wondering if the latest 3D DIT's can load pixart-alpha weights or other open source text2image model weights? Or it only can be trained from scratch?

LinB203 commented 3 months ago

Trained from scratch.

RanXu2000 commented 3 months ago

Thanks，Another question I'd like to ask, why is the 3d DIT's out_channels set to be the same as in_channels, instead of pred sigma as before, i.e. out_channels = 2*in_channels