img2img Batch Tool

[AIMusicExperiment]

Hi Alex. is there any plan to implement a batch mode any time soon? Almost all of my work in Stable diffusion involves animation, often with thousands of frames. I can't currently leverage the power of multi-gpu implementation on StableSwarm, with these huge animation generations because with a batch mode I would have to generate my videos frame by frame with manually advancing the animation nodes individually, between each frame. That more than negates the speed advantage that the quicker generation would afford me. Thanks for your thoughts on this and for all the hard work that you and your team have been doing, hopefully you can someday bring this awesome speed enhancement to those of us that really need it. . .

[mcmonkey4eva]

There are three queue-batch modes currently:

• 1: just set images high and generate a bunch of images with one set of settings
• 2: hit generate button many times and let them all queue up
• 3: Under tools, select Grid Generator, and use that to generate a wide batch based on a predefined list of settings (eg a bulk set of prompts or whatever is pretty easy to do via grid gen)

For video animation, you'd need very specifically a way to batch img2img a folder-full of source images, which is a specific case not yet covered by those options, but a tool for that could be added. For video animation I feel like the tools could be taken a lot further (just look at deforum! it's got ten million parameters and functions), which is an entire project of its own, but small parts of it along the way like a bulk img2img tool could easily be added.

If there's any specific tools beyond just init-image-input-batcher you'd want, that wouldn't be necessarily massive video-animation-focused-projects of their own, feel free to suggest.

[AIMusicExperiment]

https://www.youtube.com/watch?v=WIGqDDfOVHk

[mcmonkey4eva]

If (- IF! -) simply incrementing a primitive node works, you can:

• Use that workflow
• go to swarm, go to tools, select Grid Generator
• set the primitive as an axis, set its values to 1, 2, .., 25 (grid gen will automatically interpret this syntax to mean generate 1 image for each value from 1 to 25) That way it will loop through all 25 and shove the results together in a folder.

A lil jank but would achieve the hypothetical goal, with automatic GPU splitting.

however I do not think it is the case that incrementing the node suffices - that animation shown in the video looks to be linearly self-referential (ie, every frame depends on the content of the prior frame, meaning you cannot split it in parallel across multiple GPUs as that would cause it to lose temporal context).

This is unfortunately a very difficult case to solve in any context, not just limited to here - the animation being built from a linear process means it's naturally resistant to parallelization, regardless of UI/toolset work. With that setup, you cannot generate the next frame until the previous one is done.

---

btw you don't need to manually enable every node - leaving them disabled implicitly uses whatever value is in the workflow, turning them on is only needed if you want to customize them while on the generate tab.

---

For examples of what can be parallelized easily here:

• Video2Video, in the form of img2img on a frame-by-frame basis. As each frame can be processed independently, that's easily split into parallel jobs and run across as many GPUs as you have.
• Separate sections. You can eg trigger the generation of one part of a video on GPU, and then an entirely separate section of video on another GPU, so long as those two sections don't depend on one another (ie you intend to simply hard-cut between them).
• Any other technique by which you can avoid generations being dependent on the prior frame. If a version of that is able to generate frames from scratch (without dependence) it should work fine.
• I can imagine a hypothetical animation tool that works by generating a set of independent keyframes spaced a bit apart (say one frame for each 5 seconds) wherein those keyframes don't have to have accurate temporal coherence due to being spaced out (and thus, can be generated independently on separate GPUs!), and then the space between them is filled in with temporal coherence checks (ie: there's a "round 2" of image generations that are dependent on the results of "round 1" of generations, but are otherwise independent in sections and thus able to be split between GPUs)

[mcmonkey4eva]

For bulk frame overriding, the new Image Edit Batch tool should handle that quite nicely.

[mcmonkey4eva]

Pardon the indecisive edits above, I was going to reformat this as an open-ended discussion (as the highly actionable part was added, and past that it's kinda open-ended) then remembered https://github.com/Stability-AI/StableSwarmUI/discussions exists and so further discussion of the topic should continue there.

More issues can be made if/when we find more actionable feature improvements to make here.