search

Picsart-AI-Research / Text2Video-Zero

[ICCV 2023 Oral] Text-to-Image Diffusion Models are Zero-Shot Video Generators
https://text2video-zero.github.io/
Other
3.91k stars 336 forks source link

Missing motion Motion Dynamics Latent Codes in Text-To-Video with Pose Control #38

Open huytuong010101 opened 1 year ago

huytuong010101 commented 1 year ago

Hi Picsart-AI-Research team. Thank you for useful project, you result is really impressive. In your code, you only use Motion Dynamics Latent Codes in Text-To-Video Can you tell me why you not apply this method to Text-To-Video with Pose/Edge Control? Thank you,

rob-hen commented 1 year ago

Hi,

in text-to-video, there is no a-priori motion information, in contrast to the other setup, e.g. edge conditioning. We thus used non-trivial motion dynamics only for text-to-video while for the other setups, we set it to constant and let the conditions drive the video.

huytuong010101 commented 1 year ago

Thank u for you reply, i get it.