[feature] Implement `function_call` response interface

[rgbkrk]

Is the feature request related to a problem?

With OpenAI, the function calling messages look like this:

[
  { role: "system", content: "You are a data engineer" },
  { role: "user", content: "Show me the top 5 most popular products" },
  {
    role: "assistant",
    content: None,
    function_call: { // Model creates a function call with a name and arguments.
      name: "sql",
      arguments:
        '{\n  "query": "SELECT product_name, COUNT(*) as popularity FROM products GROUP BY product_name ORDER BY popularity DESC LIMIT 5"\n}',
    },
  },
  {
    role: "function", // The response from the caller that the model expects (see chatlab)
    content:
      "|    | product_name   |   count |\n|---:|:---------------|--------:|\n|  2 | Water          |     431 |\n|  0 | Pedialyte      |     434 |\n|  4 | Sprite         |     599 |\n|  7 | Dr. Pepper     |     714 |\n|  1 | Gatorade       |     783 |",
    name: "sql",
  },
  {
    role: "assistant",
    content:
      "The top 5 most popular products are:\n\n1. Gatorade - 783 orders\n2. Dr. Pepper - 714 orders\n3. Sprite - 599 orders\n4. Pedialyte - 434 orders\n5. Water - 431 orders",
  },
];

Gorilla puts the function call in content to be parsed instead:

[
  { role: "system", content: "You are a data engineer" },
  { role: "user", content: "Show me the top 5 most popular products" },
  { role: "assistant", content: 'sql.query("select count(*) from products")' },
];

Describe the solution you'd like

Instead of getting back an assistant message with content I have to parse, I'd love to get the function_call object back with name and arguments to work with.

Additional context

• Allowing custom servers in chatlab

[ShishirPatil]

Thanks for raising this @rgbkrk and the detailed example! We made this design choice thinking it might be better this way especially in those scenarios where you might want to exec() the code. Though you are right that it would be nice to even have the outputs OAI compatible. Will try to get to it tomorrow!

[nathanmargaglio]

Greetings @ShishirPatil ! Any update on this?

I'm also curious about this design decision, if you care to explain the thought process. I know I'm not too keen on letting an LLM run arbitrary code via exec(), and the function_call-like response makes it easy to control which functions are being called, validate arguments, etc. without adding too much coding overhead. I'm just wondering if there's some aspect to this I'm missing or if there's another approach you'd suggest.

[HAL9KKK]

Thanks for raising this @rgbkrk and the detailed example! We made this design choice thinking it might be better this way especially in those scenarios where you might want to exec() the code. Though you are right that it would be nice to even have the outputs OAI compatible. Will try to get to it tomorrow!

Hello, do you have any news @ShishirPatil about this updateing? Thanks!

[rgbkrk]

Importantly for me, I want the functions to be able to run in any language because there are times when I do server side python, rust, or javascript. The agents are basically the same across and I give them the same description of tools. The implementation changes based on the environment but the manifest remains the same.

[nathanmargaglio]

Any updates on this? Or maybe recommendations for alternative approaches?

I definitely appreciate this project for what it offers, but the function_call-like responses just seems like the more obvious approach to take, so I'm guessing there's a project somewhere which is comparable to gorilla but does it this way.

Alternatively, I would appreciate hearing the reasoning for the approach this project has taken instead. I'd assume there's a technical reason for it, so maybe an explanation would change my mind on this approach altogether.

[rgbkrk]

For the moment, I recommend using the functionary model: https://github.com/MeetKai/functionary

You can run it on Linux GPUs with vLLM and on Macs with the Llama Cpp Inference setup.