量化后的ChatGlm3-6B使用open_api部署tools功能无法使用

[LiuGuBiGu]

• 现象： 使用open_api部署后，测试tools功能，请求体如下

  {
  "functions": {
      "random_number_generator": {
          "name": "random_number_generator",
          "description": "Generates a random number x, s.t. range[0] <= x < range[1]",
          "params": [
              {
                  "name": "seed",
                  "description": "The random seed used by the generator",
                  "type": "int",
                  "required": true
              },
              {
                  "name": "range",
                  "description": "The range of the generated numbers",
                  "type": "tuple[int, int]",
                  "required": true
              }
          ]
      },
      "get_weather": {
          "name": "get_weather",
          "description": "Get the current weather for `city_name`",
          "params": [
              {
                  "name": "city_name",
                  "description": "The name of the city to be queried",
                  "type": "str",
                  "required": true
              }
          ]
      }
  },
  "model": "chatglm3-6b",
  "messages": [
      {
          "role": "user",
          "content": "帮我查询北京的天气如何"
      }
  ]
  }

预期：

• 希望返回function的相关信息，例如

  "function_call": {
                  "name": "get_weather",
                  "arguments": "{\"city_name\": \"北京\"}"
              }

  实际：

• 实际返回如下

  {
  "id": "chatcmpl",
  "model": "default-model",
  "object": "chat.completion",
  "created": 1703834923,
  "choices": [
      {
          "index": 0,
          "message": {
              "role": "assistant",
              "content": "很抱歉，作为一个人工智能助手，我无法实时查询天气信息。您可以使用手机应用或搜索引擎查询最新的北京天气。"
          },
          "finish_reason": "stop"
      }
  ],
  "usage": {
      "prompt_tokens": 12,
      "completion_tokens": 30,
      "total_tokens": 42
  }
  }

补充：使用同样的请求参数访问非量化chatglm3-6B，可以正确返回相关信息

{
    "model": "chatglm3-6b",
    "object": "chat.completion",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "get_weather\n ```python\ntool_call(city_name='北京')\n```",
                "name": null,
                "function_call": {
                    "name": "get_weather",
                    "arguments": "{\"city_name\": \"北京\"}"
                }
            },
            "finish_reason": "function_call"
        }
    ],
    "created": 1703835057,
    "usage": {
        "prompt_tokens": 285,
        "total_tokens": 307,
        "completion_tokens": 22
    }
}

[LiuGuBiGu]

但是使用cli运行看起来又可以正常使用

Welcome to ChatGLM.cpp! Ask whatever you want. Type 'clear' to clear context. Type 'stop' to exit.

System   > Answer the following questions as best as you can. You have access to the following tools:
{
    "random_number_generator": {
        "name": "random_number_generator",
        "description": "Generates a random number x, s.t. range[0] <= x < range[1]",
        "params": [
            {
                "name": "seed",
                "description": "The random seed used by the generator",
                "type": "int",
                "required": true
            },
            {
                "name": "range",
                "description": "The range of the generated numbers",
                "type": "tuple[int, int]",
                "required": true
            }
        ]
    },
    "get_weather": {
        "name": "get_weather",
        "description": "Get the current weather for `city_name`",
        "params": [
            {
                "name": "city_name",
                "description": "The name of the city to be queried",
                "type": "str",
                "required": true
            }
        ]
    }
}
Prompt   > 帮我查询北京的天气如何
ChatGLM3 > get_weather
```python
tool_call(city_name='北京')

Function Call > Please manually call function get_weather and provide the results below. Observation > 温度 5度，天气晴，湿度 50 ChatGLM3 > 北京的天气查询结果如下：当前气温为5度，天气晴朗，湿度为50%。

[Longhao-Chen]

我也遇到了这个问题，请问有什么新的进展吗？