Tools calling support

[paoloski97]

I have tried a variety of models that support function calls, none of them gives the correct answer using LM Studio. Since it uses an 'API compatible with that of OpenAI it should accept the tools key, but it doesn't seem to recognize it. Am I doing something wrong or does LM Studio not support function calls?

Code:

import requests

data = {
    'model': 'functionary-small-v3.2', # model name here is the value of argument "--model" in deploying: server_vllm.py or server.py
    'messages': [
        {
            "role": "user",
            "content": "What is the weather for Istanbul?"
        }
    ],
    'tools':[ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]
}

response = requests.post("http://localhost:1234/v1/chat/completions", json=data, headers={
    "Content-Type": "application/json",
    "Authorization": "Bearer xxxx"
})

# Print the response text
print(response.text)

[faev999]

I had the same problem and couldn't find a solution. So I switched to ollama given that it implements tool usage.

[yagil]

We're about to start a beta for this. If you're interested, please fill out this google form: https://docs.google.com/forms/d/e/1FAIpQLSfqpRunKgv2ui4_CWyeQ88gFtOG6CEqFSbpRrHWgJjs2mIodw/viewform?usp=sf_link

[vuminhquang]

Hi @yagil , Any update on this? Thank you

[khanhj]

i submitted the form, i can help test the beta version. BTW your discord server does not exist?