Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
Adds support for image "batching" to the workflow editor.
The batch API is currently in use by the linear UI, but until now there was no way for users to interact with it directly. This PR adds a new node called Image Batch, which queues a graph for each of its images.
How it works
Add an Image Batch node, drag images into it and connect the Image output into any other node(s).
When you click Invoke, the special Image Batch node is removed from the graph and a batch config is created from its list of images and all its outgoing connections. The backend gets the batch config and processes it, enqueuing a separate graph for each image in the list.
Examples
Single Batch -> Single Node
The simplest case is using a batch on a single node. Here's a batch that resizes 5 images to 200x200 thumbnails.
Results:
This results in 5 graphs, each containing a single resize node with one of the 5 images in the batch list. Note the images are 200x200 pixels.
Single Batch -> Multiple Nodes
You can also use a single batch on multiple nodes. This batch resizes the image to 200x200 pixels, then pastes this thumbnail version back on top of the full size version of the image.
Results:
This also results in 5 graphs, each of which contains a resize and paste node. In each graph, each node gets the same input image of the 5 images in the batch list. The graph does not get any of the other images in the batch.
The batch node can have any number of outgoing connections from its image output - all of them get the same image.
Multiple Batches -> Single/Multiple nodes
When multiple batches are used, they are combined such that all permutations are run (e.g. the product of batches is taken).
Results:
In this case, the product of the two batches is 15 graphs. Each image of the 3-image batch is used as the base image, and a thumbnail of each tiger is pasted on top of it.
No Pairwise Batching
When using multiple batches, they are always multiplied - never paired.
Imagine you had two batches of 5 images. As described in the multiplicative example above, you'd get 5 images * 5 images = 25 graphs.
Pairwise batching would instead take the first image from each of the two batches and use them together in each graph.
While the server's batching API supports this, it is not implemented in the UI just yet. It's much more complicated to make a UI for pairwise batching, so we'll evaluate use-cases for it and can explore a UI for it in the future.
No Dynamic Inputs
You cannot dynamically generate a batch's image collection. It must be provided up-front.
Other Field Types
This PR implements support for image batching, any field type is possible so long as we can provide a UI for it. For example, a String Batch could queue multiple graphs for a list of prompts, or a Model Batch could queue a graph for every model.
Image batching is the biggest impact here, so we've added it first. We can add other field types in the future, depending on demand. The system supports it; it's just a matter of building a solid UI/UX.
Using a Board for an Image Batch
For now, you will need to manually select all images in a board and drag them in. We'll look into ways to use a board directly in the future.
Versus Iterate Nodes
Batching serves a similar use-case to iterate nodes, and can replace them in any situation where Iterate nodes do not need their input collection to be dynamic.
Going forward, we'll suggest batching to iterate nodes for situations where there are no dynamic inputs.
Combining Batch Outputs
One thing you might like to do with batches is combine the outputs of a batch. This is not feasible for now, sorry.
Other Changes
You may now drag images into image collection fields directly.
Image collection field validation is updated to support min and max item counts.
Special was added to the Classification enum. This indicates that the UI had some special handling for the node. Maybe this should just be named batch. All it does is add a green lightning bolt icon to the node. Suppose we could also skip this classification and instead match on node type.
Related Issues / Discussions
Many offline discussions.
Closes #7034
Closes #5098 (use an upscaling workflow with an Image Batch node)
QA Instructions
My main concern is the field validation changes. Previously, field validation was mostly concerned with required fields having a value. Now, we also may need to check collection sizes. It's possible I missed something and fields render as invalid w/ red colors, when they are actually fine.
Besides fuzzing field validation, try out the Batch API.
I just realized the Invoke button's tooltip does not calculate the number of graphs it will queue. I'll do that in a followup.
Merge Plan
n/a
Checklist
[x] The PR has a short but descriptive title, suitable for a changelog
Summary
Adds support for image "batching" to the workflow editor.
The batch API is currently in use by the linear UI, but until now there was no way for users to interact with it directly. This PR adds a new node called
Image Batch, which queues a graph for each of its images.How it works
Add an
Image Batchnode, drag images into it and connect theImageoutput into any other node(s).When you click Invoke, the special
Image Batchnode is removed from the graph and a batch config is created from its list of images and all its outgoing connections. The backend gets the batch config and processes it, enqueuing a separate graph for each image in the list.Examples
Single Batch -> Single Node
The simplest case is using a batch on a single node. Here's a batch that resizes 5 images to 200x200 thumbnails.
Results:
This results in 5 graphs, each containing a single resize node with one of the 5 images in the batch list. Note the images are 200x200 pixels.
Single Batch -> Multiple Nodes
You can also use a single batch on multiple nodes. This batch resizes the image to 200x200 pixels, then pastes this thumbnail version back on top of the full size version of the image.
Results:
This also results in 5 graphs, each of which contains a resize and paste node. In each graph, each node gets the same input image of the 5 images in the batch list. The graph does not get any of the other images in the batch.
The batch node can have any number of outgoing connections from its image output - all of them get the same image.
Multiple Batches -> Single/Multiple nodes
When multiple batches are used, they are combined such that all permutations are run (e.g. the product of batches is taken).
Results:
In this case, the product of the two batches is 15 graphs. Each image of the 3-image batch is used as the base image, and a thumbnail of each tiger is pasted on top of it.
No Pairwise Batching
When using multiple batches, they are always multiplied - never paired.
Imagine you had two batches of 5 images. As described in the multiplicative example above, you'd get 5 images * 5 images = 25 graphs.
Pairwise batching would instead take the first image from each of the two batches and use them together in each graph.
While the server's batching API supports this, it is not implemented in the UI just yet. It's much more complicated to make a UI for pairwise batching, so we'll evaluate use-cases for it and can explore a UI for it in the future.
No Dynamic Inputs
You cannot dynamically generate a batch's image collection. It must be provided up-front.
Other Field Types
This PR implements support for image batching, any field type is possible so long as we can provide a UI for it. For example, a
String Batchcould queue multiple graphs for a list of prompts, or aModel Batchcould queue a graph for every model.Image batching is the biggest impact here, so we've added it first. We can add other field types in the future, depending on demand. The system supports it; it's just a matter of building a solid UI/UX.
Using a Board for an Image Batch
For now, you will need to manually select all images in a board and drag them in. We'll look into ways to use a board directly in the future.
Versus Iterate Nodes
Batching serves a similar use-case to iterate nodes, and can replace them in any situation where Iterate nodes do not need their input collection to be dynamic.
Going forward, we'll suggest batching to iterate nodes for situations where there are no dynamic inputs.
Combining Batch Outputs
One thing you might like to do with batches is combine the outputs of a batch. This is not feasible for now, sorry.
Other Changes
Specialwas added to theClassificationenum. This indicates that the UI had some special handling for the node. Maybe this should just be named batch. All it does is add a green lightning bolt icon to the node. Suppose we could also skip this classification and instead match on node type.Related Issues / Discussions
Many offline discussions.
Image Batchnode)QA Instructions
My main concern is the field validation changes. Previously, field validation was mostly concerned with required fields having a value. Now, we also may need to check collection sizes. It's possible I missed something and fields render as invalid w/ red colors, when they are actually fine.
Besides fuzzing field validation, try out the Batch API.
I just realized the Invoke button's tooltip does not calculate the number of graphs it will queue. I'll do that in a followup.
Merge Plan
n/a
Checklist