stavsap / comfyui-ollama

Apache License 2.0
374 stars 34 forks source link

Add one round chat node using ollama.Client.chat #18

Closed LingXuanYin closed 3 weeks ago

LingXuanYin commented 4 months ago

Add two new nodesOllamaOneRoundChat and OllamaOneRoundChatAdvance ollama.Client.chat Add retry when generate or chat get error Check response or response["message"]["content"] before return Check response key when using Using logging instead print Rename file CompfyuiOllama.py -> ComfyUI_Ollama.py

stavsap commented 4 months ago

@LingXuanYin Thanks for the PR. by the way you need to add the node name to js file that trigger the model list refresh. its in the web/js path.

why the addition of the retry?

LingXuanYin commented 4 months ago

@stavsap Thank you for reviewing my PR. I have added the new node name to web/js/OllamaNode.js

  1. When using the OllamaGenerateAdvancenode through the ComfyUI API, I encounter an error message 'KeyError: "eval_duration", which I cannot reproduce. It often appears without any reason. Adding retry is to facilitate the operation and maintenance of API users. Although WebUI users only need to click again when encountering errors, such errors will increase the workload of API users in operation and maintenance.
  2. When using LLM, it is common to receive a portion of output that is meaningless or does not meet the format requirements of the user. In the retry loop, judgment for such situations is added to reduce the operational workload of API users, and to facilitate all users to determine whether the LLM output meets the format requirements. I am still working on this part. Currently, only empty LLM outputs are evaluated.

sorry for using machine translation, my English is not good

stavsap commented 4 months ago

by the way what is the difference between the generate api and chat? isnt generate works the same? in your code you dont store the messages so the chat history is reset on each call. in OllamaGenerateAdvance node there is an option to store context so it acts similar to chat history no?