search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
56.06k stars 5.93k forks source link

[Feature Request] ControlNet lowvram #82

Closed jn-jairo closed 1 year ago

jn-jairo commented 1 year ago

The Mikubill/sd-webui-controlnet has a nice lowvram optimization, I can run it on stable-diffusion-webui in my 2GB GPU, but I cannot run the ControlNet from here because I get an out of memory error.

I tried to understand how it works to make that optimization in this repo, but I guess I'm not smart enough for that, I know it keeps things on cpu and load to gpu only when needed and send back to cpu after used, but I couldn't figure out how to make it in this repo.

It would be nice if someone smart like you could add this optimization here 😁

teldon commented 1 year ago

Having a lowvram mode for controlnet would be amazing as I am about 250MB short at worst when trying to run controlnet.

jn-jairo commented 1 year ago

I tried the t2i-adapter with the models from https://huggingface.co/webui/ControlNet-modules-safetensors and it works on my GPU, so I'll use t2i-adapter from now on, it looks like the better option for my lack of vram.

comfyanonymous commented 1 year ago

I fixed some controlnet vram related issue a few days ago so your OOM error might be fixed.