search

firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to integrate, test, and deploy sophisticated AI features to Firebase or Google Cloud.
Apache License 2.0
607 stars 74 forks source link

[Docs] which models supports tools? #661

Closed xster closed 3 weeks ago

xster commented 1 month ago

Trying out a few ollama models. But they fail with

Model "ollama/qwen2" does not support tools, but some tools were supplied to generate(). Please call generate() without tools if you would like to use this model.

Even phi3 or gemma2 doesn't support tools. Well which ones do support tools?

MichaelDoyle commented 1 month ago

Hey @xster. Currently ollama does not support tool (function) calling. Models that do support it include:

Tool (or function) calling is generally dependent on availability from the model provider. It's my understanding that ollama does not provide a tool calling API. I have seen there are other frameworks out there that provide "wrappers" that implement tool calling for any model so long as it can provide structured output (i.e. json).

This is something we could discuss and consider either internally (perhaps as a plugin?) or as a community plugin.

xster commented 1 month ago

Not supporting ollama is ok. It should just be documented. Either here https://firebase.google.com/docs/genkit#4_give_llms_tools or in a breakout page about tool use (though I notice tool doesn't have a page)

xster commented 1 month ago

Though it seems like langchain somehow supports tools via ollama

MichaelDoyle commented 1 month ago

I see. Yes, the docs for tool calling can definitely be improved -- we'll prioritize that. I agree, on the page you linked to, things are worded in such a way that it appears that "tool calling" is a Genkit provided feature, and not dependent on model specific capabilities. The Generate Content page does mention tool calling is a model specific feature: https://firebase.google.com/docs/genkit/models#tools_and_function_calling, but its not very helpful.

I am not sure it would be prudent for us to list model by model what features are supported; model specific documentation is the source of truth for that. But maybe there is a way for us to make "supported features" more easily discoverable in the framework. You can see this pretty easily in the code if you know where to look. Here is ollama: https://github.com/firebase/genkit/blob/main/js/plugins/ollama/src/index.ts#L72-L75

Wrt langchain, yes, they are doing this via an abstraction/wrapper layer. We could consider the same, perhaps via middleware. I'll open a FR for this.

xster commented 1 month ago

Also see https://ollama.com/blog/tool-support

MichaelDoyle commented 3 weeks ago

Fixed via https://github.com/firebase/genkit/pull/809