search

if-ai / ComfyUI_IF_AI_tools

ComfyUI-IF_AI_tools is a set of custom nodes for ComfyUI that allows you to generate prompts using a local Large Language Model (LLM) via Ollama. This tool enables you to enhance your image generation workflow by leveraging the power of language models.
https://ko-fi.com/impactframes
MIT License
512 stars 39 forks source link

readme

ComfyUI-IF_AI_tools

ComfyUI-IF_AI_tools

ComfyUI-IF_AI_tools is a set of custom nodes to Run Local and API LLMs and LMMs, Features OCR-RAG (Bialdy), nanoGraphRAG, Supervision Object Detection, supports Ollama, LlamaCPP LMstudio, Koboldcpp, TextGen, Transformers or via APIs Anthropic, Groq, OpenAI, Google Gemini, Mistral, xAI and create your own charcters assistants (SystemPrompts) with custom presets and muchmore

Prerequisite Installation (Poppler)

To ensure compatibility and functionality with all tools, you may need poppler for PDF-related operations. Use scoop to install poppler on Windows:

Step 1: Install scoop (if not already installed)

If you haven't installed scoop yet, run the following command in PowerShell:

iwr -useb get.scoop.sh | iex

Step 2: Install poppler with scoop Once scoop is installed, you can install poppler by running:

windows 10+ istall scoop and then 

scoop install poppler

Debian/Ubuntu

sudo apt-get install poppler-utils

MacOS

brew install poppler

check is working

pdftotext -v

Install Ollama

You can technically use any LLM API that you want, but for the best expirience install Ollama and set it up.

To install Ollama models just open CMD or any terminal and type the run command follow by the model name such as

ollama run llama3.2-vision

If you want to use omost 

ollama run impactframes/dolphin_llama3_omost

if you need a good smol model

ollama run ollama run llama3.2

Optionally Set enviromnet variables for any of your favourite LLM API keys "XAI_API_KEY", "GOOGLE_API_KEY", "ANTHROPIC_API_KEY", "MISTRAL_API_KEY", "OPENAI_API_KEY" or "GROQ_API_KEY" with those names or otherwise it won't pick it up you can also use .env file to store your keys

Features

[NEW] nanoGraphRAG, [NEW] OCR-RAG ColPali & ColQwen (Bialdy), [NEW] Supervision Object Detection Florence2
[NEW] Endpoints xAI, Transformers, [NEW] IF_Assistants System Prompts with Reasoning/Reflection/Reward Templates and custom presets

ollama run impactframes/llama3_ifai_sd_prompt_mkr_q4km:latest

ollama run impactframes/ifai_promptmkr_dolphin_phi3:latest

https://huggingface.co/impactframes/llama3_if_ai_sdpromptmkr_q4km

https://huggingface.co/impactframes/ifai_promptmkr_dolphin_phi3_gguf

Installation

  1. Open the manager search for IF_AI_tools and install

Install ComfyUI-IF_AI_tools -hardest way

  1. Navigate to your ComfyUI custom_nodes folder, type CMD on the address bar to open a command prompt, and run the following command to clone the repository:

      git clone https://github.com/if-ai/ComfyUI-IF_AI_tools.git

    OR

  2. In ComfyUI protable version just dounle click embedded_install.bat or type CMD on the address bar on the newly created custom_nodes\ComfyUI-IF_AI_tools folder type 

      H:\ComfyUI_windows_portable\python_embeded\python.exe -m pip install -r requirements.txt

    replace C:\ for your Drive letter where you have the ComfyUI_windows_portable directory

  3. On custom environment activate the environment and move to the newly created ComfyUI-IF_AI_tools

      cd ComfyUI-IF_AI_tools
  python -m pip install -r requirements.txt

Related Tools

Videos

AIFuzz made a great video usining ollama and IF_AI tools

AIFuzz

Also Future thinker @ Benji Thankyou both for putting out this awesome videos

Future Thinker @Benji

Example using normal Model

ancient Megastructure, small lone figure 'A dwarfed figure standing atop an ancient megastructure, worn stone towering overhead. Underneath the dim moonlight, intricate engravings adorn the crumbling walls. Overwhelmed by the sheer size and age of the structure, the small figure appears lost amidst the weathered stone behemoth. The background reveals a dark landscape, dotted with faint twinkles from other ancient structures, scattered across the horizon. The silent air is only filled with the soft echoes of distant whispers, carrying secrets of times long past. ethereal-fantasy-concept-art, magical-ambiance, magnificent, celestial, ethereal-lighting, painterly, epic, majestic, dreamy-atmosphere, otherworldly, mystic-elements, surreal, immersive-detail' _IF_prompt_Mkr__00011_ _IF_prompt_Mkr__00012_ _IF_prompt_Mkr__00010_ _IF_prompt_Mkr__00014_

TODO

Support

If you find this tool useful, please consider supporting my work by:

:IFAItools_comfy