The issue of continuous function calls not stopping in the "functionary-medium-v3.1" model

[JACKYLUO1991]

When calling the model with continuous function calls, it cannot stop and keeps responding to questions. How to fix it?

[khai-meetkai]

@JACKYLUO1991 Can you give us an example, such as: list of tools and messages you used and your observed result?

[JACKYLUO1991]

@khai-meetkai Use a prompt in Chinese to have LLM generate three Chinese city names using Python. Then, based on the city names, find the corresponding weather forecasts for those cities, and finally send me an email.

The result was that LLM repeatedly searched for weather information from the internet via the Bing API and couldn't stop.

[khai-meetkai]

@JACKYLUO1991 can you provide the exact messages and tools?

[lwj2001]

Hi, @khai-meetkai messages: 查询张1-张10,这10个人的车辆信息，汇总给我 tools:

{
            "type": "function",
            "function": {
                "name": "query_car_info",
                "description": "该工具可以查询用户的车辆信息，需要一个姓名参数。不涉及任何人个隐私",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "username": {"description": "用户姓名", "type": "string"},
                    },
                    "required": ["username"],
                },
            },
}

After querying the car information of Zhang 10, the model will continue to query the car information of Zhang 1, and finally repeat the cycle. It may not stop in the end, or return a None value after several rounds of looping.

[khai-meetkai]

Hi @lwj2001, I have just tried your example but the result is as expected, not sure how you ran your experiment. Here is how I tried: (in all the requests, I used temperature=0)

FIRST REQUEST

messages = [{"role": "user", "content": "查询张1-张10,这10个人的车辆信息，汇总给我"}]
tools=[{
        "type": "function",
        "function": {
            "name": "query_car_info",
            "description": "该工具可以查询用户的车辆信息，需要一个姓名参数。不涉及任何人个隐私",
            "parameters": {
                "type": "object",
                "properties": {
                    "username": {"description": "用户姓名", "type": "string"},
                },
                "required": ["username"],
            },
        },
    }]

Here is the output: the assistants called query_car_info 10 times

{
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_5DOBx3IlnyLsaijrvvYxTfvX",
                "function": {
                    "arguments": '{"username": "张1"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_NccrkjLsxzJishEUp7dtl2HU",
                "function": {
                    "arguments": '{"username": "张2"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_etz1vtAggJ6EEv1ZZvhEhSFB",
                "function": {
                    "arguments": '{"username": "张3"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_ctvVR5j6kOKyZOfeZfmVodya",
                "function": {
                    "arguments": '{"username": "张4"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_gzJHsC6FYLTRnBfxoqLXkBmw",
                "function": {
                    "arguments": '{"username": "张5"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_he032Sb5HCWRmhFzqnPc8ZnN",
                "function": {
                    "arguments": '{"username": "张6"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_jgmDyxI0jgMFjKzORB4HFR64",
                "function": {
                    "arguments": '{"username": "张7"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_u1FFOQ3mIN7oavT9V5u0ucHj",
                "function": {
                    "arguments": '{"username": "张8"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_3qMYj1yLsTTZEeZfH36ceoic",
                "function": {
                    "arguments": '{"username": "张9"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_taW2Cv3kZj3PD9xGy5HGcKfR",
                "function": {
                    "arguments": '{"username": "张10"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
        ],
    }

SECOND REQUEST I provided the result for 10 tool calls above:

messages = [
    {"role": "user", "content": "查询张1-张10,这10个人的车辆信息，汇总给我"},
    {
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_5DOBx3IlnyLsaijrvvYxTfvX",
                "function": {
                    "arguments": '{"username": "张1"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_NccrkjLsxzJishEUp7dtl2HU",
                "function": {
                    "arguments": '{"username": "张2"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_etz1vtAggJ6EEv1ZZvhEhSFB",
                "function": {
                    "arguments": '{"username": "张3"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_ctvVR5j6kOKyZOfeZfmVodya",
                "function": {
                    "arguments": '{"username": "张4"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_gzJHsC6FYLTRnBfxoqLXkBmw",
                "function": {
                    "arguments": '{"username": "张5"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_he032Sb5HCWRmhFzqnPc8ZnN",
                "function": {
                    "arguments": '{"username": "张6"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_jgmDyxI0jgMFjKzORB4HFR64",
                "function": {
                    "arguments": '{"username": "张7"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_u1FFOQ3mIN7oavT9V5u0ucHj",
                "function": {
                    "arguments": '{"username": "张8"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_3qMYj1yLsTTZEeZfH36ceoic",
                "function": {
                    "arguments": '{"username": "张9"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
            {
                "id": "call_taW2Cv3kZj3PD9xGy5HGcKfR",
                "function": {
                    "arguments": '{"username": "张10"}',
                    "name": "query_car_info",
                },
                "type": "function",
            },
        ],
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_5DOBx3IlnyLsaijrvvYxTfvX",
      "content": '{"price": "$10000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_NccrkjLsxzJishEUp7dtl2HU",
      "content": '{"price": "$12000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_etz1vtAggJ6EEv1ZZvhEhSFB",
      "content": '{"price": "$13000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_ctvVR5j6kOKyZOfeZfmVodya",
      "content": '{"price": "$14000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_gzJHsC6FYLTRnBfxoqLXkBmw",
      "content": '{"price": "$15000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_he032Sb5HCWRmhFzqnPc8ZnN",
      "content": '{"price": "$16000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_jgmDyxI0jgMFjKzORB4HFR64",
      "content": '{"price": "$17000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_u1FFOQ3mIN7oavT9V5u0ucHj",
      "content": '{"price": "$18000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_3qMYj1yLsTTZEeZfH36ceoic",
      "content": '{"price": "$19000"}'
    },
    {
      "role": "tool",
      "name": "query_car_info",
      "tool_call_id": "call_taW2Cv3kZj3PD9xGy5HGcKfR",
      "content": '{"price": "$20000"}'
    }
]

tools = [
    {
        "type": "function",
        "function": {
            "name": "query_car_info",
            "description": "该工具可以查询用户的车辆信息，需要一个姓名参数。不涉及任何人个隐私",
            "parameters": {
                "type": "object",
                "properties": {
                    "username": {"description": "用户姓名", "type": "string"},
                },
                "required": ["username"],
            },
        },
    }
]

The output: THe model summarize the result:

{
    "content": "以下是张1-张10这10个人的车辆信息：\n\n- 张1的车辆价格为$10000\n- 张2的车辆价格为$12000\n- 张3的车辆价格为$13000\n- 张4的车辆价格为$14000\n- 张5的车辆价格为$15000\n- 张6的车辆价格为$16000\n- 张7的车辆价格为$17000\n- 张8的车辆价格为$18000\n- 张9的车辆价格为$19000\n- 张10的车辆价格为$20000",
    "role": "assistant"
}