search

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.
Apache License 2.0
10.88k stars 973 forks source link

Why is VAE designed to be causal, what's the advantage of it? #276

Open awei-6 opened 1 month ago

awei-6 commented 1 month ago

Usually we design causal models because we want to use autoregressive generation afterward, but as diffusion is generating in parallel, why is VAE designed to be causal? What's the intuition behind this design?

qqingzheng commented 1 month ago

This can enable VAE to support both image and video encoding and decoding.