Feature request: Abillity to to provide what workflow the remote instances should be using

[Dobidop]

I'm looking into being able to being able to do parts of/different workflows on remote instances to speed up generation.

I got this working by changing the input for the workflow to be JSON and set the prompt in "remote_chain" to be the workflow instead of the prompt. I currently load the JSON from the custom node "LoadJsonFromUrl" from comfyui-art-venture and plug it into the remote chain start.

To use this I just export the current workflow as an API workflow and set it to load into the input of the remote start node. In this exported workflow for the other instance of ComfyUI I might have set it to handle the latents 1 and 2 in the batch. In the workflow in the UI instance I might then set it to handle latents 3 and 4 from the batch.

Changes to the workflow easily makes the remote instance fail, but these minor changes seem to be ok.

I have tested this in two cases now where it works well:

• Upscaling via Tiled IP adapter, letting them handle their own tiles from the batch of latents.
• StableVideo

Here is the changes I made... So it would need to be done by someone who actually knows what they're doing instead of my tinkering.

class QueueRemoteChainStart:
    def __init__(self):
        pass
    @classmethod
    def INPUT_TYPES(s):
        return {
            "required": {
                "workflow": ("JSON",{"default": ["current"]}), #changed the input

---

def chain_start(self, workflow, trigger, batch, seed, prompt):
        remote_chain = {
            "seed": seed+batch,
            "batch": batch,
            "prompt": workflow, #changed to use the workflow instead of the  prompt
            "current_seed": seed+batch,
            "current_batch": batch,
            "job_id": f"{get_client_id()}-{int(time.time()*1000*1000)}"

[city96]

To use this I just export the current workflow as an API workflow and set it to load into the input of the remote start node. In this exported workflow for the other instance of ComfyUI I might have set it to handle the latents 1 and 2 in the batch.

I wonder if it would be possible to automate this. Like add a node that just returns the current workflow, which can then be altered further by other nodes (the JSON can be passed as a string, so even regex could work). Then it should be possible to do per-client overrides for anything, not just the batch size.

Upscaling via Tiled IP adapter, letting them handle their own tiles from the batch of latents

This actually sounds really cool! The upscale is the slowest part so seamlessly distributing it would be amazing.

Here is the changes I made... So it would need to be done by someone who actually knows what they're doing instead of my tinkering.

Don't worry, I also have zero real coding experience, so your tinkering is as good as mine lol. I really need to get back to this project sometime, especially now that I have at least 2 decent-ish GPUs I could distribute stuff across. Hopefully I'll have more free time in the coming weeks.

[Dobidop]

I was looking into creating a node which could provide the instance URL to allow it to use that to drive some switch logic, allowing automatic selection of batches, i.e. if in server http://127.0.0.1:1111/ do batch 1 and 2, and if on http://127.0.0.1:2222/ do batch 3 and 4. Would mean that in some cases the workflow wouldn't need to be different, the same one could be used.

A current issue I have with my current setup is that I have to move the workflow file to the input folder to be able to select it, I can't get it saved and loaded automatically.

[city96]

Okay, I rewrote just about the whole thing lol. You should be able to patch in whatever workflow you want... I think.

It'd be nice to have nodes to edit the loaded workflow directly in the UI, but I didn't get to that yet. The default save path is just outputs/workflow, you'll have to move the json to the inputs folder for it to pick it up, but that's about it.

[Dobidop]

Haha, madlad!

Work better now. Added a short post on reddit about the tiled IP adapter upscale concept. https://www.reddit.com/r/comfyui/comments/18z5rwb/distributed_gpu_upscale_via_tiled_ip_adapter/

As for the loading of the workflow, for now I just use the URL json loader "LoadJsonFromUrl" from comfyui-art-venture to load it from any valid folder.

[city96]

Added a short post on reddit about the tiled IP adapter upscale concept.

That's pretty cool. I still got some bugs to iron out but once that's done I plan to do my own network distributed tiled sampler lol. It was the reason why I got back to this project again. Upscaling to 10000x6000 is a tiny bit slow on a single card.

This is full of jank and can't really automate this since I'd need a new node for loading the latent over the network properly

This doesn't seem too hard to do. I already got the load image ones, might as well try and add this too.

[city96]

Check the readme ;D

[Dobidop]

Dude, great work! With all of these recent changes it really feels like we can get even more great use out of this node pack. Looking forward to implementing this in my workflows.