search

if-ai / ComfyUI-IF_AI_tools

ComfyUI-IF_AI_tools is a set of custom nodes for ComfyUI that allows you to generate prompts using a local Large Language Model (LLM) via Ollama. This tool enables you to enhance your image generation workflow by leveraging the power of language models.
https://ko-fi.com/impactframes
366 stars 27 forks source link

IMG2 Prompt (Text) not working for me #34

Closed ALBIHANY closed 1 month ago

ALBIHANY commented 1 month ago

What am i doing wrong here !!! ? Why i can't i get image to prompt, i also tried with other models i can only get txt2image work

image

ThomasRoyer24 commented 1 month ago

What am i doing wrong here !!! ? Why i can't i get image to prompt, i also tried with other models i can only get txt2image work

image

You're using the wrong assistant. IF_PromptMKR is an assistant for text to image. IF_PromptMKR_IMG should be used for image to text. You can look here to see what each assistant does: https://github.com/if-ai/ComfyUI-IF_AI_tools/blob/main/presets/assistants.json

if-ai commented 1 month ago

What am i doing wrong here !!! ? Why i can't i get image to prompt, i also tried with other models i can only get txt2image work

image

As @ThomasRoyer24 said but besides that it looks like you have the image still muted I muted it by default and there is a note over it saying you need to select the node and press CTRL+M to unmute also for working with images you need a multimodal model like llava, moondream or use the anthropic APIs I like haiku a lot and gpt4-o is also very strong. but in ollama and textgen you need a moondream or llava-llama3 or llava base models

ALBIHANY commented 1 month ago

Thank you very much for your comments and instructions, it's highly appreciated really, i also downloaded LLaVA 3 and the result are really impressive i will try the other models you mentioned, but i'm surprised you can also use gpt4-o already i thought it's not accessible at the moment, i will also check that out

ALBIHANY commented 1 month ago

@if-ai May i ask why moondream doesn't get me any results do i need to change some parameters? other models (such as llava-llama3) using the same parameters worked very well, thank you in advance.

Untitled