search

rashadphz / farfalle

🔍 AI search engine - self-host with local or cloud LLMs
https://www.farfalle.dev/
Apache License 2.0
2.21k stars 166 forks source link

500: 'Message' object has no attribute 'tool_calls' #59

Open anzestrela opened 1 week ago

anzestrela commented 1 week ago

I am running a slightly modified version of farfalle, to be able to use a custom OpenAI compatable API I changed llm.llm = LiteLLM(model=llm.llm.model) to llm.llm = LiteLLM(model=llm.llm.model, api_base="http://192.168.73.78:5000/v1") it generates the answer but when trying to generate related_queries it errors with: 500: 'Message' object has no attribute 'tool_calls' I know that the version of llama3-8b I am using supports function calling, as it is working with openwebUI.

arsaboo commented 5 days ago

Yeah, looks like llama3 does not work very well. Try other models.