An Ollama client! Run your local models with AutoGen using a dedicated client class.
One of the key features of this library (and which is still very much experimental) is the support for tool calling. This is done "manually" by injecting tools into the prompt and translating between AutoGen's tool call objects and text messages (updated to also support Ollama's native tool calling). The use of tool calling will be described in further detail below but essentially you should be able to get up and running with it as it stands without customising the text injected to support it.
This manual tool calling approach and the actual text injected is an initial attempt to handle tool calling so if you can help improve it, please do!
I'll use the client with some notebooks and local models and summarise the results in another comment.
To run the code, you'll need to install the ollama and fix-busted-json packages (these will be automatically installed when this is merged and you install through pip install pyautogen[ollama]):
pip install ollamapip install fix-busted-json
Updated to include Ollama's native tool calling (just released, v0.3.0 Ollama library)
An Ollama client! Run your local models with AutoGen using a dedicated client class.
One of the key features of this library (and which is still very much experimental) is the support for tool calling. This is done "manually" by injecting tools into the prompt and translating between AutoGen's tool call objects and text messages (updated to also support Ollama's native tool calling). The use of tool calling will be described in further detail below but essentially you should be able to get up and running with it as it stands without customising the text injected to support it.
This manual tool calling approach and the actual text injected is an initial attempt to handle tool calling so if you can help improve it, please do!
I'll use the client with some notebooks and local models and summarise the results in another comment.
To run the code, you'll need to install the
ollamaandfix-busted-jsonpackages (these will be automatically installed when this is merged and you install throughpip install pyautogen[ollama]):pip install ollamapip install fix-busted-jsonUpdated to include Ollama's native tool calling (just released, v0.3.0 Ollama library)
Related issue numbers
Checks
(3056)