search

SHI-Labs / Smooth-Diffusion

Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models arXiv 2023 / CVPR 2024
https://shi-labs.github.io/Smooth-Diffusion/
MIT License
319 stars 8 forks source link

Does this work for video generation? #10

Closed Njasa2k closed 8 months ago

Njasa2k commented 8 months ago

Does Smooth-Diffusion work for video generation or something like animatediff?

JiayiGuo821 commented 8 months ago

Hi @ninjasaid2k, thanks for pointing out the promising direction!

Through image interpolation, some animation-like results can be achieved.

For video generation via smooth diffusion, there are several possible ways: 1) Generate start and end frames and interpolate them. 2) Replace SD with smooth diffusion for existing T2V works that use SD as the base model.

We will keep exploring and updating once there are some new results.