search

phidatahq / phidata

Build AI Assistants with memory, knowledge and tools.
https://docs.phidata.com
Mozilla Public License 2.0
11.25k stars 1.67k forks source link

Question about OpenAILike usage #137

Closed antonsapt4 closed 6 months ago

antonsapt4 commented 7 months ago

Saw this LM-Studio Cookbook, 

from phi.assistant import Assistant
from phi.llm.openai.like import OpenAILike
from phi.tools.duckduckgo import DuckDuckGo

assistant = Assistant(
    llm=OpenAILike(base_url="http://localhost:1234/v1"),
    tools=[DuckDuckGo()],
    show_tool_calls=True,
)
assistant.print_response("Whats happening in France? Summarize top stories with sources.", markdown=True)

Is it because LM-Studio OpenAI API provided is supporting tools?

Since I'm using colab wondering how OpenAILike works and tried with Groq endpoint llm=OpenAILike(base_url="https://api.groq.com/openai/v1", api_key="gsk_xxxxxx", model="mixtral-8x7b-32768"),

it said: tools is not supported with this model. Just want to get my head around it.

And kudos with phi-data it really make opensource model usable in very clear way without many workaround!

ysolanky commented 7 months ago

Hello! Groq currently does not support function calling. You can check out our Gorq cookbook and use Groq directly.

Since many providers use OpenAI API format, we use OpenAILike to access them.

ashpreetbedi commented 7 months ago

Thanks @antonsapt4, as @ysolanky pointed out Groq doesnt support tools yet. As soon as they do we should have that available as well :)

antonsapt4 commented 7 months ago

Thanks @ysolanky @ashpreetbedi. Yes just read on groq website that the endpoint is not supported function calling.

After playing around, looks like only Ollama with openhermes model that can do tools calling, do you know what local model that can use the tools / function calling?

LM Studio even tho using the same OpenHermes (https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF) looks like still cannot call tools using the code above.

Really appreciate if anyone had any experience using different model beside Ollama OpenHermes to have comparison.

ashpreetbedi commented 6 months ago

@antonsapt4 sadly currently we've only seen openhermes to be decent-ish at tool calling. We're still very early in the function calling journey, even so much that most LLM providers dont support it yet. We'll see this become mainstream in the next few months and you'll get more models/LLM providers supporting it

ashpreetbedi commented 6 months ago

@antonsapt4 try out hermes2 for function calling, it is very very good for a 7b model: https://github.com/phidatahq/phidata/tree/main/cookbook/hermes2