Omost tool doesn't work

[blueraincoatli]

I thought I've intalled the node correctly, and downloaded the needed models, but the Omost tool doesn't generate the json code, please help~

You are using ollama as the engine, no API key is required.
Error processing canvas: Response does not contain codes!
You are a helpful AI assistant to compose images using the `Canvas` class.

Here's an image description that includes the input request:

**Global Description:** High-tech warrior, futuristic armor, advanced technology, sci-fi setting

**Local Descriptions:**

* Head: Helmet with visor, augmented reality display, sleek design
* Body: Armored suit, metallic plating, energy-absorbing material, flexible joints
* Arms: Retractable blades, grappling hook, energy shield generator
* Legs: Jet-powered propulsion system, retractable landing gear
* Hands: Grip-enhanced gloves, multi-tool attachment
* Feet: Magnetic boots, hoverboard capabilities

**Tags:** Female warrior, high-tech armor, futuristic setting, sci-fi, advanced technology, augmented reality display, energy-absorbing material, flexible joints, jet-powered propulsion system, grappling hook, energy shield generator, grip-enhanced gloves, multi-tool attachment, magnetic boots, hoverboard capabilities.

**Atmosphere:** Futuristic cityscape, neon lights, towering skyscrapers, advanced machinery

**Style:** Hyper-realistic, detailed textures, metallic sheen, vibrant colors

**Quality Meta:** High-resolution image, 8K resolution, HDR, advanced lighting techniques

**HTML Web Color Name:** #FFC080 (Goldenrod)

Please let me know if you need any further modifications or details.
OmostTool execute method returning: {'error': "Response does not contain codes!\nYou are a helpful AI assistant to compose images using the `Canvas` class.\n\nHere's an image description that includes the input request:\n\n**Global Description:** High-tech warrior, futuristic armor, advanced technology, sci-fi setting\n\n**Local Descriptions:**\n\n* Head: Helmet with visor, augmented reality display, sleek design\n* Body: Armored suit, metallic plating, energy-absorbing material, flexible joints\n* Arms: Retractable blades, grappling hook, energy shield generator\n* Legs: Jet-powered propulsion system, retractable landing gear\n* Hands: Grip-enhanced gloves, multi-tool attachment\n* Feet: Magnetic boots, hoverboard capabilities\n\n**Tags:** Female warrior, high-tech armor, futuristic setting, sci-fi, advanced technology, augmented reality display, energy-absorbing material, flexible joints, jet-powered propulsion system, grappling hook, energy shield generator, grip-enhanced gloves, multi-tool attachment, magnetic boots, hoverboard capabilities.\n\n**Atmosphere:** Futuristic cityscape, neon lights, towering skyscrapers, advanced machinery\n\n**Style:** Hyper-realistic, detailed textures, metallic sheen, vibrant colors\n\n**Quality Meta:** High-resolution image, 8K resolution, HDR, advanced lighting techniques\n\n**HTML Web Color Name:** #FFC080 (Goldenrod)\n\nPlease let me know if you need any further modifications or details.", 'prompt': 'Describe a female worrior in hightech armor', 'llm_response': "You are a helpful AI assistant to compose images using the `Canvas` class.\n\nHere's an image description that includes the input request:\n\n**Global Description:** High-tech warrior, futuristic armor, advanced technology, sci-fi setting\n\n**Local Descriptions:**\n\n* Head: Helmet with visor, augmented reality display, sleek design\n* Body: Armored suit, metallic plating, energy-absorbing material, flexible joints\n* Arms: Retractable blades, grappling hook, energy shield generator\n* Legs: Jet-powered propulsion system, retractable landing gear\n* Hands: Grip-enhanced gloves, multi-tool attachment\n* Feet: Magnetic boots, hoverboard capabilities\n\n**Tags:** Female warrior, high-tech armor, futuristic setting, sci-fi, advanced technology, augmented reality display, energy-absorbing material, flexible joints, jet-powered propulsion system, grappling hook, energy shield generator, grip-enhanced gloves, multi-tool attachment, magnetic boots, hoverboard capabilities.\n\n**Atmosphere:** Futuristic cityscape, neon lights, towering skyscrapers, advanced machinery\n\n**Style:** Hyper-realistic, detailed textures, metallic sheen, vibrant colors\n\n**Quality Meta:** High-resolution image, 8K resolution, HDR, advanced lighting techniques\n\n**HTML Web Color Name:** #FFC080 (Goldenrod)\n\nPlease let me know if you need any further modifications or details."}
==================
IF_AI_tool_output:
==================
Error in omost_tool: Response does not contain codes!
You are a helpful AI assistant to compose images using the `Canvas` class.

Here's an image description that includes the input request:

**Global Description:** High-tech warrior, futuristic armor, advanced technology, sci-fi setting

**Local Descriptions:**

* Head: Helmet with visor, augmented reality display, sleek design
* Body: Armored suit, metallic plating, energy-absorbing material, flexible joints
* Arms: Retractable blades, grappling hook, energy shield generator
* Legs: Jet-powered propulsion system, retractable landing gear
* Hands: Grip-enhanced gloves, multi-tool attachment
* Feet: Magnetic boots, hoverboard capabilities

**Tags:** Female warrior, high-tech armor, futuristic setting, sci-fi, advanced technology, augmented reality display, energy-absorbing material, flexible joints, jet-powered propulsion system, grappling hook, energy shield generator, grip-enhanced gloves, multi-tool attachment, magnetic boots, hoverboard capabilities.

**Atmosphere:** Futuristic cityscape, neon lights, towering skyscrapers, advanced machinery

**Style:** Hyper-realistic, detailed textures, metallic sheen, vibrant colors

**Quality Meta:** High-resolution image, 8K resolution, HDR, advanced lighting techniques

**HTML Web Color Name:** #FFC080 (Goldenrod)

Please let me know if you need any further modifications or details.
==================
IF_AI_tool_output:
==================

==================
IF_AI_tool_output:
==================
Describe a female worrior in hightech armor
==================
IF_AI_tool_output:
==================
null
!!! Exception during processing!!! 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_omost\omost_nodes.py", line 325, in render_canvas
    numpy2pytorch(imgs=[OmostCanvas.render_initial_latent(canvas_conds)]),
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_omost\lib_omost\canvas.py", line 370, in render_initial_latent
    conds[0]["color"]
    ~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

Prompt executed in 12.36 seconds

[if-ai]

Omost is not a vision model so it can't see the image you need to disable it. Mute Controls on the READ note, I left a note saying so under the you can use vision models like gpt4 o mini to describe or LlaVa without omost

[blueraincoatli]

Thanks for your answer, I did try to mute the image load node but the result is the same, the response output port outputs a detail prompt, but the tool_output port is null. It seems the LLM_response is not right.

[if-ai]

I forgot to put this but because omost continually improve the prompt in many iterations the stopping string has to be none. Otherwise stops too early also you need an specific omost model to work with omost tool you can install for ollama using this command ollama run impactframes/dolphin_llama3_omost I also made a video about working with this video about omost

[blueraincoatli]

Yes it works after I set the stop string 'none' and change the specific omost model, thank you!