replicate / comfyui-replicate

Run Replicate models as nodes in ComfyUI
https://replicate.com/explore
MIT License
33 stars 6 forks source link

Only the last image is output when processing a list of images with a Replicate NODE #2

Open shamanicvocalarts opened 5 days ago

shamanicvocalarts commented 5 days ago

Only the last image is output when processing a list of images with A Replicate node

Description: When sending a list of images to the Replicate model for processing, all images are correctly processed according to the logs and the Replicate backend. However, only the last image in the list is output to the subsequent nodes (e.g., VHS Combine node and Save Image node), while the previous images are not passed along as expected.

Steps to Reproduce Set up nodes:

Use the Replicate model node (e.g., philz1337x/clarity-upscaler) to process a list of images. Connect the output of the Replicate model node to both the VHS Combine node and the Save Image node. Execute the batch:

Provide a list of images to the Replicate model node for processing. Execute the nodes. Expected behavior:

All processed images from the list should be output to the VHS Combine node. All processed images should be individually saved by the Save Image node. Actual behavior:

Only the last image in the list is output to the VHS Combine node and saved by the Save Image node. Previous images are not passed along or saved.

Possible Cause The issue might be related to how the output from the Replicate model node is being handled or passed to subsequent nodes. It seems that only the final image in the processed list is being output, while the previous images are discarded. Steps Taken to Resolve Retried with smaller batches and confirmed that the issue persists. Ensured that all nodes are correctly connected and configured. Suggested Solution Investigate and ensure that the node handling logic can correctly pass all images in the list to subsequent nodes rather than only the last image.

fofr commented 4 days ago

Ah, the issue here is that the models do not support batch inputs, it'll make a single call with the first image.

Models with a single input and multiple outputs work.

This is a limitation that I will probably need to work around by batching up API calls when passing multiple inputs. 🤔

How would you want this to work?

shamanicvocalarts commented 4 days ago

so the behaviour I noticed is that it would make multiple calls ( on larger image batches/lists it had a timeout error that unfortunately I didnt log, but with smaller batches this was not an issue ) 1 by 1 & execute properly on the backend. but only 1 image would return from the node at the end of the operation ( I assumed the last one, but it could be the first one as you state here) . I think the expected behaviour observed with a local node, when input is batched images/ image list & output can take batched images / image lists, is to hold all the images & output as a batch/ list.

I will quickly set up a test workflow to demonstrate expected behaviour from a local node as compared to a replicate node & post here

fofr commented 4 days ago

This is at least doing more than I expected it to, so it could be simpler to build out than I'd planned.

shamanicvocalarts commented 3 days ago

LocalImageBatchVSReplicateImageBatch.json

attached is a workflow using a local node (Ultimate SD Upscale) next to replicate clarity node demonstrating expected behaviour. input image directory with X frames ( I suggest 10 for faster response ) & note how the Ultimate Upscale node outputs a batch which is received properly by VHS video combine

shamanicvocalarts commented 3 days ago

also note the image batch to list node before the replicate node, it seems most local nodes handle this by default ? cant say for certain as I am very new to comfyUI node development but I would say handling image batch & image lists natively would be expected behaviour