Function Calling API Calls not OpenAI-conform?

[localmind-ai]

Hi @khai-meetkai - we're currently evaluating the larger functionary model based on Llama 3.1 70B and are wondering if the vLLM-based API implementation is correct. For the same API call (an example I have taken from here), I get the following responses:

Functionary's Response

{
   "id":"cmpl-190a0024ca304cfb9730201284bb56dd",
   "object":"chat.completion",
   "created":1723319870,
   "model":"meetkai/functionary-medium-v3.1",
   "choices":[{
         "index":0,
         "message":{
            "role":"assistant",
            "tool_call_id":null,
            "content":null,
            "name":null,
            "function_call":{
               "name":"outings",
               "arguments":"{\"area\": \"Amsterdam\"}"
            },
            "tool_calls":null
         },
         "finish_reason":"function_call"
      }],
   "usage":{
      "prompt_tokens":978,
      "total_tokens":993,
      "completion_tokens":15
   }
}

Discrepancy between OpenAI Functions and Tools

If I'm not mistaken, functions are deprecated and tool calls used instead. See this example snippet:

{
    finish_reason: 'tool_calls',
    index: 0,
    logprobs: null,
    message: {
        content: null,
        role: 'assistant',
        function_call: null,
        tool_calls: [
            {
                id: 'call_62136354',
                function: {
                    arguments: '{"order_id":"order_12345"}',
                    name: 'get_delivery_date'
                },
                type: 'function'
            }
        ]
    }
}

Is this going to be considered in the template at some point? Thanks a lot!

[jeffrey-fong]

Although functions are deprecated in OpenAI API, they are still supporting it as of now. Thus, we are also supporting it currently. If you would like to use tools, you can change to making the request with tools instead of functions (your API call is calling with functions, not tools). We support API requests with either tools or functions, just like OpenAI API.

[localmind-ai]

Okay, I see! Thank you a lot @jeffrey-fong, I just double checked and verified that I used the old way of API calling which is why I got the "old-way response". Doing it the more recent way gave me the correct results. Thanks, will close this now.