latest version is very slow after first run?

[ranpo]

EDIT: Sorry, I tried something else and found the "problem" for the slow speed. The "keep alive" is off by default and after turning it on the speed is back to normal as how it was in previous version. I guess in previous version it's on by default?

But as you can see on the log, the first run was fast (16 secs, this includes the time to load the model). subsequent run, even with model loading and unloading (because keep alive is off), it took around 100 secs.

---

Hello. I just upgraded comfyui, ComfyUI-IF_AI_tools, and ollama. When I try it, it ran normally when I first executed it. Then subsequent execution, without changing the image, it's getting extremely slow.

I tried to run llava directly on ollama (not comfyui) , the speed is normal (very quick).

The node I use are IF Chat Prompt, and IF Image Prompt. All settings are default except the model (llava 7b) and profile (none and IF_PromptMKR_IMG)

Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  4.49it/s]
Global Step: 840001
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
Leftover VAE keys ['model_ema.decay', 'model_ema.num_updates']
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 16.24 seconds
got prompt
[rgthree] Using rgthree's optimized recursive execution.
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  5.51it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 138.04 seconds
got prompt
[rgthree] Using rgthree's optimized recursive execution.
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  5.57it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 119.72 seconds
got prompt
[rgthree] Using rgthree's optimized recursive execution.
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00,  5.62it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 133.17 seconds

Also, is the node supposed to keep executed even when nothing is connected to it?

Thank you :)

[if-ai]

Thank you, yes the previous version never unloaded the model, if you want it to go faster you can use keep_alive to be like before but it takes more memory.

[ranpo]

Alright, thank you very much for a great custom node :)

[if-ai]

thank you