search

gokayfem / ComfyUI_VLM_nodes

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation
Apache License 2.0
297 stars 23 forks source link

Structured Output breaking. #79

Open RangerFrank opened 2 months ago

RangerFrank commented 2 months ago

!!! Exception during processing !!! Traceback (most recent call last): File "D:\AI\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\suggest.py", line 611, in keyword_extract parsed_response = json.loads(response) ^^^^^^^^^^^^^^^^^^^^ File "json__init__.py", line 346, in loads File "json\decoder.py", line 337, in decode File "json\decoder.py", line 353, in raw_decode json.decoder.JSONDecodeError: Unterminated string starting at: line 3 column 15 (char 17)

MilitantHitchhiker commented 2 months ago

!!! Exception during processing !!! Traceback (most recent call last): File "D:\AI\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\suggest.py", line 611, in keyword_extract parsed_response = json.loads(response) ^^^^^^^^^^^^^^^^^^^^ File "json__init__.py", line 346, in loads File "json\decoder.py", line 337, in decode File "json\decoder.py", line 353, in raw_decode json.decoder.JSONDecodeError: Unterminated string starting at: line 3 column 15 (char 17)

MilitantHitchhiker commented 2 months ago

Over and over and over. keeps breaking workflows and preventing continuous processing without babysitting the node.

gokayfem commented 2 months ago

which llm you are using?

MilitantHitchhiker commented 2 months ago

llava 1.6 7b.Q5 mistral

gokayfem commented 2 months ago

maybe you can use more json output oriented llm, like nous hermes type llms, you can find their gguf files in huggingface

MilitantHitchhiker commented 2 months ago

Unfortunately the hermes models kind of suck for this purpose, the llava base models give significantly better outputs, any chance of adding some basic formatting error handling into the node?