Update Plan

[continue-revolution]

Announcement

---

Content

This PR serves as the plan and announcement for future updates. It will never be merged or closed.

This extension will serve as the A1111 version of video generation toolkit. For the time being, the majority of video generative models are based on AnimateDiff, but I don't believe that AnimateDiff is the final destination - there must be better model architectures.

A large version bump normally means new model support satisfying the following criteria:

• open-sourced video generative diffusion model, not limited to AnimateDiff itself
• proven to be high-performance / very popular (typically more than 1k GitHub stars or highly phrased within the community)
• you are encouraged to submit a feature request with new models and / or new features, but you need to pursuade me that it is actually needed. Whenever you encounter a new work, learn to doubt and experiment before getting too excited - the majority of academic works are 100% garbage.
• you are more encouraged to participate in coding and submit a PR to accelerate the development processed. I am paid to do something completely unrelated to video diffusion, and the development progress can never be guaranteed given my current situation.

A list of new models that might be worth supporting

• [ ] Animate Anyone (Not open-sourced yet, but seems very exciting. They used AnimateDiff)
• [x] Stable Video Diffusion (Most likely combined with a PR to A1111 + AnimateDiff -> SVD VAE)

A small version bump normally means new community features not “officially” supported but I think it is worth having (may come from myself or somewhere else).

A list of community features that might be worth supporting

• [ ] MasaCtrl
• [ ] tokenflow
• [x] prompt travel w/ ip-adapter
• [ ] controlnet per-frame weights
• [ ] improve i2v (dive into denoising strength, probably see here) & i2v per-frame weights - is it possible to completely lock 0 & -1 frame? #251
• [ ] motion weight
• [ ] region prompt: from CLI / ComfyUI / MultiDiffusion / LatentCouple
• [x] #236
• [ ] #259
• [ ] make sure that cli features work, especially upscale, refine, tile-upscale, stylize - monitor cli update
• [ ] check some comfy workflows on reddit, discord and twitter
• [ ] #213
• [ ] batch size (KohakuBlueleaf will PR)

[continue-revolution]

Hi @zappityzap

I am going to invite you to join this repo as a collaborator. I appreciate your contribution so far. In the future, you can

• work in your own branch in this repo instead of your fork. Please create your own branch in the format of <some prefix of your own choice>/<your own branch name> (such as zap/output) so that I can know that it's your branch.
• close any issue that you think is resolved
• your PR still need my approval, but in 99% cases I will merge instantly

[zappityzap]

Thank you.

[continue-revolution]

later update plan will be maintained in a private doc, subject to my own interest. This pr will be closed and we will no longer update.