Usage

[p0mad]

Hi Thanks for your nice work!

I'm new to this repo, and there is not a general knowledge share on the page on how to do the gpu share on Comfy! Please be paitaient if i'm asking noob questions.

1- i have installed a ComfyUI on a Cloud based ubuntu image which has a T4 GPU. I also have two 8Gb VRAM Gpus on other ubuntu PC. How can i configure the Comfy to make use of the PC GPUs?

2- Do i need to install Comfy on Both Cloud and PC? ( Cloud already has Comfy installed)

3- Does it work over SSH? and it works for ControlNet?

4- Can you please explain steps on how to set it up in both devices?

5- Does it works on Cloud + Windows?

Thanks Best regards

[city96]

Hey! This was originally a random one-weekend project so the documentation (and the code for that matter) is pretty shoddy lol. There's also a rewrite branch, which is a bit different and might be more stable. I don't think I have an image of that but here's one I quickly threw together:

1- i have installed a ComfyUI on a Cloud based ubuntu image which has a T4 GPU. I also have two 8Gb VRAM Gpus on other ubuntu PC. How can i configure the Comfy to make use of the PC GPUs?

You need to pick a machine that will be your "main" PC - this is where you run the webui you actually want to access. The other ones are only used over the API, but you still need a way to reach them.

So for instance, let's say you have two GPUs in your ubuntu PC. You would start your instance as normal, this would be accessible on 127.0.0.1:8188. Then you start a second one, but force it to use the second GPU (put CUDA_VISIBLE_DEVICES=1 in front of your command). This second one will have to be on a different port, let's say 127.0.0.1:8288. Now you can use the queue_on_remote node to start the workflow on the second GPU instance while also running on your main one. I recommend you get this working before trying to add the cloud one, because debugging it locally should be easier.

2- Do i need to install Comfy on Both Cloud and PC? ( Cloud already has Comfy installed)

Yes, but it's also a bit more annoying since all your checkpoints/LoRAs will have to have the exact same name to work. (This is why there is also a windows/posix switch in the node - not even the path separator can differ). The whole script relies on just sending the current workflow to the remote UI's API as if it were a normal workflow, so it's pretty fragile in general.

3- Does it work over SSH?

Yes! look up "SSH port forwarding". You could forward the cloud instance to be available on 127.0.0.1:8388, or do the reverse and make your two local UIs accessible on the cloud machine's localhost.

and it works for ControlNet?

I think it'd only work if you run it locally, or somehow sync the image folders between your local/remote instance. (When you load up a controlnet image, it gets saved to disk so the workflow can load it). You could try rsync or even something simple like sharing the entire ComfyUI/input folder between the two by attaching it via sshfs on the client machines.

4- Can you please explain steps on how to set it up in both devices?

1. Install ComfyUI on both machines, also download any models you might need on both.

2. Pick a machine as your main one, I recommend the local one. Make sure it can access the remote one - either via regular or ssh port forwarding.

3. Set up the remote nodes. You should have one node to start the workflow on the remote machine, while the rest of the workflow will run as normal on the local one. See the image above or in the readme for an example.

5- Does it works on Cloud + Windows?

It should, just set the OS correctly in the node to avoid the path issues mentioned above.

Hope this helps trying to get my jank to work. Just ask if something breaks/sounds confusing/etc.

[p0mad]

@city96 Thank you so much for the detailed guide, i'll try that

I'm a liitle bit confused about the computations. I'm not sure which parts of my workflow could get into another Machine(port forwarded to the main PC)?

My workflow for a Single GPU is something like this: 1- The input directory is loaded as path for images folder 2- Each image goes into some preprocessing ControlNet nodes 3- Preprocessed images then goes into the Controlnet load and apply model 4- ControlNet output gets into the Ksampler and then to the decoder and save image 5- This process repeats for all images into that directory

P.S one of the Controlnets have TemporalNet model!

I'm not sure how can i distribute these Nodes onto multiple GPU/Comfy instances!?

So instead of your Empty latent, I give {input image + VAE} from the Load Checkpoint to the {VAE Encode} and then the conncets it to latent image and instead of Positive prompt connect directly to the ksampler Positive, The Positive prompt connects to CN apply and then to the ksampler Positive!

input and output images are 512*512. Seed and Prompt is always Fix. just Input image and ControlNet[in regards] gets different!

I have a laptop with very Low-end GPU[2GB], and i have a T4 GPU [16GB] on cloud, Two 1080 on a ubuntu PC, two 2080 GPU on Win10 PC!

Can you please guide me how to utilize all of them?

Thanks Best regards

[city96]

Seed and Prompt is always Fix. just Input image and ControlNet[in regards] gets different!

Hmm, that is indeed a different usecase. So if I understand it right you have a "fixed" workflow (fixed seed and everything) but need the input image to change for some kind of batch processing. Are you using it for animations or something else?

I've experimented with this kind of distributed batch processing too but I never published that code anywhere. If you can wait until the weekend then I might be able fix up and publish some of that old code for you!

[p0mad]

you have a "fixed" workflow (fixed seed and everything)

Yes, I need to run the same workflow for multiple images[Batch], but thats not the first thing to try. Inside each workflow, i also have multiple preprocessing on the input image, including CN preprocessings and BBox detection and cropping which takes time and also needs to get distributed into multiple GPUs!

In other word the Priority goes with distributing preprocessing on multiple GPUs for a single input image. and then Sure i would be glad to wait for a week to distribute the batch processing!

As mentioned above:

My workflow for a Single GPU is something like this: 1- The input directory is loaded as path for images folder 2- Each image goes into some preprocessing ControlNet nodes 3- Preprocessed images then goes into the Controlnet load and apply model 4- ControlNet output gets into the Ksampler and then to the decoder and save image 5- This process repeats for all images into that directory

This file contain a simple workflow

Please change the checkpoint and also install Missing Nodes using manager.

Thanks in advance

[city96]

@p0mad

Hey. I uploaded the distribution script. I cleaned it up a bit but it's pretty rudimentary: https://github.com/city96/ComfyUI_NetDist/tree/main/mass-process

You need to edit the example config to match what you need - it should have comments on what does what. I think it should work with the preprocessor as long as you have it installed on all the machines you're going to use for this. You can just launch it with server.py --conf your_job.yaml and it should connect to all the machines in the config. (Don't accidentally launch the wrong "server.py" lol, I did that once while writing it.)

Ping me if it doesn't work. I can write a better explanation of how to run it tomorrow!

[Dobidop]

Here is a basic NetDist workflow.

This is the 'primary' workflow and no workfow or UI is required for the 'secondary', located at port 8189 in my case.

The server just need to be started. The secondary server will use the same workflow as the primary, but with a different seed.