The GPU memory usage is too high

[peki12345]

About 60G? This is too scary, can it be optimized?

[8600862]

torch.cuda.OutOfMemoryError: CUDA out of memory.

[microbenh]

How to run pipeline in several GPUs, like 4*4090

[JPlin]

You can try this to in two GPUs: https://huggingface2.notion.site/How-to-split-Flux-transformer-and-run-inference-aa1583ad23ce47a78589a79bb9309ab0

[c-steve-wang]

You can try this to in two GPUs: https://huggingface2.notion.site/How-to-split-Flux-transformer-and-run-inference-aa1583ad23ce47a78589a79bb9309ab0

Could you kindly provide the script of how this method works with the main.py?

[microbenh]

You can try this to in two GPUs: https://huggingface2.notion.site/How-to-split-Flux-transformer-and-run-inference-aa1583ad23ce47a78589a79bb9309ab0

It do not work. The transformer need 24GB, and controlnet 4GB, they have to be the same gpu.

[Nomination-NRB]

You can try this to in two GPUs: https://huggingface2.notion.site/How-to-split-Flux-transformer-and-run-inference-aa1583ad23ce47a78589a79bb9309ab0

It do not work. The transformer need 24GB, and controlnet 4GB, they have to be the same gpu.

It worked, I used two 3090s to get results, but inpaint was poor, and poorly followed prompt to redraw

[JPlin]

27 Fixed some bugs, now need 28GB of VRAM.

[xhinker]

I can successfully run it using torchao using about 20G VRAM, the result is great