search

if-ai / ComfyUI-IF_AI_tools

ComfyUI-IF_AI_tools is a set of custom nodes for ComfyUI that allows you to generate prompts using a local Large Language Model (LLM) via Ollama. This tool enables you to enhance your image generation workflow by leveraging the power of language models.
https://ko-fi.com/impactframes
366 stars 27 forks source link

Ollama keep_alive: #17

Closed ekondev closed 2 months ago

ekondev commented 2 months ago

By default ollama keeps the last loaded model in the vram for 5 min. Could you please add a keep_alive: 0 to flush it right after the generation and free up the vram for comfy? I've tried to add it manually but it only works on image to prompt node, prompt to prompt fails for some reason. here

if-ai commented 2 months ago

Right, good idea I will add it later thanks for the suggestion

if-ai commented 2 months ago

By default ollama keeps the last loaded model in the vram for 5 min. Could you please add a keep_alive: 0 to flush it right after the generation and free up the vram for comfy? I've tried to add it manually but it only works on image to prompt node, prompt to prompt fails for some reason. here

I added the feature now you can select to keep them on memory or by default the immediately unload after generation