TinyTerra / ComfyUI_tinyterraNodes

A selection of nodes for Stable Diffusion ComfyUI
GNU General Public License v3.0
341 stars 41 forks source link

PipeLoader reloading model unnecessarily? #58

Closed viewscreenon closed 5 months ago

viewscreenon commented 9 months ago

I don't know when this started happening, but pipeLoader seems to be unnecessarily re-loading models when any of its parameters change. In a simple setup like this: Screenshot 2023-10-16 103939 If I generate an image then change a parameter in pipeLoader (such as changing the prompt or changing empty latent size), then I generate again, it has to re-load the model. It shouldn't have to re-load when only changing something like the prompt, right?

I'm not sure when this started happening. I just came back and updated Comfy and its extensions after 2 months away.

TinyTerra commented 9 months ago

Yeah it should keep the model cached. I'm not currently able replicate this... There is a log message from base Comfy saying 'Loading 1 new model' - but it doesn't actually reload the model.

Would you mind sharing the console log?

viewscreenon commented 9 months ago

Here's the console after a model should already be loaded, I change the prompt in pipeLoader, and generate another image:

got prompt
model_type EPS
adm 0
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
missing {'cond_stage_model.text_projection', 'cond_stage_model.logit_scale'}
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
You shouldn't move a model when it is dispatched on multiple devices.
loading in lowvram mode 1702.5432691574097

I am in lowvram mode on my laptop with only 4GB vram, using SD 1.5 models. I don't know if this is related, but I found that if I left config_name in pipeLoader as "Default" I got a CUDA out of memory error. So I set it to "v1-inference".