search

THUDM / CogVideo

Text-to-video generation: CogVideoX (2024) and CogVideo (ICLR 2023)
Apache License 2.0
7.22k stars 665 forks source link

looping video #149

Open dribnet opened 3 weeks ago

dribnet commented 3 weeks ago

Feature request / 功能建议

Does anyone know if it is possible to force the video to loop seamlessly?

Motivation / 动机

Would be a great feature for posting short videos

Your contribution / 您的贡献

I might be able to implement this with some guidance would gladly contribute this back to the project.

zRzRzRzRzRzRzR commented 3 weeks ago

Could you describe it in more detail? Are you trying to extend the video by continuously generating 6-second segments and stitching them together into a longer video? :)

dribnet commented 3 weeks ago

Well - I meant something like this 😉

trimmed

If interested, I can try to clean this up a bit and submit it as a pull request as a pipeline option or maybe just a standalone demo using callback_on_step_end.

zRzRzRzRzRzRzR commented 3 weeks ago

That sounds interesting! Are you connecting the first and last frames of a video to create a 6-second looping animation? If so, I’d be happy to contribute a pull request (PR) as a tool! Looking forward to collaborating.

screan commented 1 week ago

If interested, I can try to clean this up a bit and submit it as a pull request as a pipeline option or maybe just a standalone demo using callback_on_step_end.

looks great! came looking for the same

dribnet commented 1 week ago

looks great! came looking for the same

Thanks - my version is a quick hack, but I cleaned it up a bit and pushed it to a with_looping branch in my fork in case anyone wants to use it as a starting point. There's a utility called cli_demo_loop.py that will create videos that more or less match at the beginning and end. Here's a few examples:

python cli_demo_loop.py --prompt "A couple are having dinner together." with_loop_02g

python cli_demo_loop.py --prompt "A woman is running on the beach in the late afternoon." with_loop_03g

python cli_demo_loop.py --prompt "A lion is drinking water." with_loop_06g

There's still some glitchiness at the seam and depending on the prompt you might get only an approximate match. For example:

python cli_demo_loop.py --prompt "A man is skiing." with_loop_04g

or it might not match at all and just insert a cut frame 🤷‍♂️

python cli_demo_loop.py --prompt "Nightfall in a metropolis." with_loop_05g

but when it does work it's pretty neat. I might come back to it and refine it or happy to discuss & improve it with anyone.