search

aigc-apps / CogVideoX-Fun

📹 A more flexible CogVideoX that can generate videos at any resolution and creates videos from images.
Apache License 2.0
486 stars 32 forks source link

Continuation of previous animation #51

Open tobiaswuerth opened 3 weeks ago

tobiaswuerth commented 3 weeks ago

Hello, love the results!

I was wondering if there is a way to continue an existing animation by providing multiple starting frames (and optionally a new end frame)? i.e. custom mask ala OSP If no, is there a plan to add something like this for inference?

Keep up the great work Thanks

bubbliiiing commented 2 weeks ago

Is it a continuation of the video?

tobiaswuerth commented 2 weeks ago

Yes, that would be the idea.

Currently, when I have 4 keyframes for example, I need to make 3 runs to get an animation from frame 1->2, 2->3, 3->4 and stitch the 3 videos together to get 1 long animation over the 4 keyframes. The issue with that is that the animation of the individual segments might be completly different (see this I made for example) and therefor doesn't really look continoues.

One solution to this could be to allow multiple starting frames (like a video, yes) such that the model can continoue the animation in a natural flow.

Another solution would be to directly allow multiple keyframes at once (i.e. all 4 in this example) for the model to create the complete animation via all 4 frames.

Is it understandable what I mean with this?

PS. I'm working with kijai/ComfyUI-CogVideoXWrapper, so I'm not sure if this is a limitation of the wrapper or the model.

Thanks for any suggestions.