chatglm3 openai 流式chat的数据建模和工具tool调用代码实现补充

[lilongxian]

Feature request / 功能建议

1，修复了openai chat system指令建模代码。 2，完善tool调用程序

Motivation / 动机

最近不少朋友说他们使用openai接口做agent tool调用时候不知道怎么调，我研究了chatglm3-6b openai接口后发现写的很好，同时我也完善了一点，为了更好的服务chatglm社区爱好者，我把补充的代码提到这里，希望能够更新到官方代码里。

Your contribution / 您的贡献

1. 修复openai chat system指令建模代码。修复的代码详情我已提在 https://github.com/THUDM/ChatGLM3/issues/1162 【chatglm3 openai流式对话接口chat数据建模的改进 #1162】 原始代码位于https://github.com/THUDM/ChatGLM3/tree/main/openai_api_demo/utils.py

修复代码：

def process_chatglm_messages(messages, tools=None): _messages = messages messages = [] msg_has_sys = False # 新加，表示system指令状态 if tools: messages.append( { "role": "system", "content": "Answer the following questions as best as you can. You have access to the following tools:", "tools": tools } ) msg_has_sys = True # system指令状态：已添加

for m in _messages:
    role, content, func_call = m.role, m.content, m.function_call
    if role == "function":
    messages.append(
        {
            "role": "observation",
            "content": content
        }
    )

    elif role == "assistant" and func_call is not None:
        for response in content.split("<|assistant|>"):
        metadata, sub_content = response.split("\n", maxsplit=1)
        messages.append(
            {
                "role": role,
                "metadata": metadata,
                "content": sub_content.strip()
            }
        )
    else:
        # # 当没有tools时候，这里role也可以是system role，仅仅运行普通的chat场景，而非agent-chat场景。
        # # 但当在agent-chat场景时，因上面已经加入了sys信息，所以这里就不再添加，将system状态复位为0。
        if role == "system" and msg_has_sys:   # 
            msg_has_sys = False  #  system状态复位
            continue   #  
        messages.append({"role": role, "content": content})
return messages

以上的改进，可以兼容agent-chat工具调用场景多轮对话，也可以用于通用的有sys指令信息的对话场景。

经过改后的agent-chat运行效果如如下： agent-chat中第一次调用 messages after process_chatglm_messages: [ {'role': 'system', 'content': 'Answer the following questions as best as you can. You have access to the following tools:', 'tools': [{'name': 'track', 'description': '追踪指定股票的实时价格', 'parameters': {'type': 'object', 'properties': {'symbol': {'description': '需要追踪的股票代码'}}, 'required': []}}, {'name': 'Calculator', 'description': '数学计算器，计算数学问题', 'parameters': {'type': 'object', 'properties': {'symbol': {'description': '要计算的数学公式'}}, 'required': []}}]}, {'role': 'user', 'content': '你好!'}, {'role': 'assistant', 'content': '你好！请问有什么我可以帮助你的吗？'}, {'role': 'user', 'content': '37乘以8加7除2等于多少？'} ] agent-chat中第二次调用 messages after process_chatglm_messages: [ {'role': 'system', 'content': 'Answer the following questions as best as you can. You have access to the following tools:', 'tools': [{'name': 'track', 'description': '追踪指定股票的实时价格', 'parameters': {'type': 'object', 'properties': {'symbol': {'description': '需要追踪的股票代码'}}, 'required': []}}, {'name': 'Calculator', 'description': '数学计算器，计算数学问题', 'parameters': {'type': 'object', 'properties': {'symbol': {'description': '要计算的数学公式'}}, 'required': []}}]}, {'role': 'user', 'content': '你好!'}, {'role': 'assistant', 'content': '你好！请问有什么我可以帮助你的吗？'}, {'role': 'user', 'content': '37乘以8加7除2等于多少？'}, {'role': 'assistant', 'content': "Calculator\n python\ntool_call(symbol='37*8+7/2')\n"}, {'role': 'observation', 'content': '299.5'} ]

2，完善tool调用程序 源码位于 https://github.com/THUDM/ChatGLM3/blob/main/openai_api_demo/api_server.py

完善代码如下：

（1）新建py脚本，用于存放tool schema信息，如下 from Tools.Calculator import Calculator

tool_class = { 'Calculator': Calculator,

'track': Track

        }

tool_def = [{"name": "track", "description": "追踪指定股票的实时价格", "parameters": {"type": "object", "properties": {"symbol": {"description": "需要追踪的股票代码"}}, "required": []}}, {"name": "Calculator", "description": "数学计算器，计算数学问题", "parameters": {"type": "object", "properties": {"symbol": {"description": "要计算的数学公式"}}, "required": []}} ]

（2）api_server.py脚本中 from Tools.tool_scan import tool_class, tool_def

定义函数： def contains_custom_function(value: str, tools: list) -> bool: for tool in tools: if value and tool["name"] in value: return True

（3）工具调用：

    # 工具调用. # Here is the handling of stream = True
    function_call = None
    if output and gen_params["tools"]:
        try:
            function_call = process_response(output, use_tool=True)
        except:
            logger.warning("Failed to parse tool call")

    # CallFunction
    if isinstance(function_call, dict):
        print("Call Function ...")
        function_call = FunctionCallResponse(**function_call)

        """----- 计算tool_response，即observation------"""
       """
        In this demo, we did not register any tools.
        You can use the tools that have been implemented in our `tools_using_demo` and implement your own streaming tool implementation here.
        Similar to the following method:
        """
        if tool_param_start_with in output:
            tool = tool_class.get(function_call.name)
            if tool:
                tool_param = json.loads(function_call.arguments).get("symbol")
                if tool().parameter_validation(tool_param):
                    observation = str(tool().run(tool_param))
                    tool_response = observation
                else:
                    tool_response = "Tool parameter values error, please tell the user about this situation."
            else:
                tool_response = "No available tools found, please tell the user about this situation."
        else:
            tool_response = "Tool parameter content error, please tell the user about this situation."

        """-------observation 计算完毕 ------"""

        if not gen_params.get("messages"):
            gen_params["messages"] = []

        gen_params["messages"].append(ChatMessage(
            role="assistant",
            content=output,  # "Calculator\n```python\ntool_call(symbol='37*8+7/2')\n```"
        ))
        gen_params["messages"].append(ChatMessage(
            role="function",  # 即：observation
            name=function_call.name,  # Calculator
            content=tool_response,  # 52.32590180780452
        ))

        print("Call Function gen_params after observation run: ", gen_params)

        # Streaming output of results after function calls
        generate = predict(model_type, gen_params)
        return EventSourceResponse(generate, media_type="text/event-stream")

这样改进的好处： （1）单独增加了Tools schema模块，与chatglm3 openai主程序分离，保证了主程序完整性，也增强了工具集开发与适配的灵活性，本工具集使用langchain技术开发。 （2）在api_server.py中将工具调用部分进行了抽象、统一的处理，保证了主程序完整性，无需用户干涉，用户仅仅需要导入自定义的tools schema即可。

[ATP-BME]

在ChatGLM3的工具调用实现中，还有一个参数是否必须提供的True/False参数。请问这个参数是否可以与langchain的实现方式对齐呢？当前的对齐似乎没有考虑这一点

[ATP-BME]

在ChatGLM3的工具调用实现中，还有一个参数是否必须提供的True/False参数。请问这个参数是否可以与langchain的实现方式对齐呢？当前的对齐似乎没有考虑这一点

尝试在现有代码的基础上改进了一下

工具定义函数中

ChatGLM3/langchain_demo/tools/Calculator.py `import abc

from typing import Type from langchain.tools import BaseTool from pydantic import BaseModel, Field

class CalculatorInput(BaseModel): calculation: str = Field(description="calculation to perform",required=True)

class Calculator(BaseTool, abc.ABC): name = "Calculator" description = "Useful for when you need to calculate math problems" args_schema: Type[BaseModel] = CalculatorInput

def __init__(self):
    super().__init__()

def _run(self, calculation: str) -> str:
    calculation = calculation.replace("^", "**")
    if "sqrt" in calculation:
        calculation = calculation.replace("sqrt", "math.sqrt")
    elif "log" in calculation:
        calculation = calculation.replace("log", "math.log")
    return eval(calculation)`

template映射中 ChatGLM3/langchain_demo/ChatGLM3.py `def _tool_history(self, prompt: str): ans = []

    tool_prompts = prompt.split(
        "You have access to the following tools:\n\n")[1].split("\n\nUse a json blob")[0].split("\n")
    tools_json = []

    for tool_desc in tool_prompts:
        name = tool_desc.split(":")[0]
        description = tool_desc.split(", args:")[0].split(":")[1].strip()
        parameters_str = tool_desc.split("args:")[1].strip()
        parameters_dict = ast.literal_eval(parameters_str)
        params_cleaned = {}

        required_list = []
        for param, details in parameters_dict.items():
            params_cleaned[param] = {'description': details['description'], 'type': details['type']}
            if details['required']:
                required_list.append(param)

        ## type properties required
        tools_json.append({
            "name": name,
            "description": description,
            "parameters": {'type':'object','properties':params_cleaned,'required':required_list}
        })
    print(tools_json)

    ans.append({
        "role": "system",
        "content": "Answer the following questions as best as you can. You have access to the following tools:",
        "tools": tools_json
    })`

调用结果 input_variables=['agent_scratchpad', 'input', 'tool_names', 'tools'] input_types={'chat_history': typing.List[typing.Union[langchain_core.messages.ai.AIMessage, langchain_core.messages.human.HumanMessage, langchain_core.messages.chat.ChatMessage, langchain_core.messages.system.SystemMessage, langchain_core.messages.function.FunctionMessage, langchain_core.messages.tool.ToolMessage]]} messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=['tool_names', 'tools'], template='Respond to the human as helpfully and accurately as possible. You have access to the following tools:\n\n{tools}\n\nUse a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).\n\nValid "action" values: "Final Answer" or {tool_names}\n\nProvide only ONE action per $JSON_BLOB, as shown:\n\n```\n{{\n "action": $TOOL_NAME,\n "action_input": $INPUT\n}}\n```\n\nFollow this format:\n\nQuestion: input question to answer\nThought: consider previous and subsequent steps\nAction:\n```\n$JSON_BLOB\n```\nObservation: action result\n... (repeat Thought/Action/Observation N times)\nThought: I know what to respond\nAction:\n```\n{{\n "action": "Final Answer",\n "action_input": "Final response to human"\n}}\n\nBegin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation')), MessagesPlaceholder(variable_name='chat_history', optional=True), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['agent_scratchpad', 'input'], template='{input}\n\n{agent_scratchpad}\n (reminder to respond in a JSON blob no matter what)'))] [{'name': 'Calculator', 'description': 'Useful for when you need to calculate math problems', 'parameters': {'type': 'object', 'properties': {'calculation': {'description': 'calculation to perform', 'type': 'string'}}, 'required': ['calculation']}}] *****Action***** {'action': 'Calculator', 'action_input': {'calculation': '34*34'}} *****Answer***** {'input': '34 * 34', 'output': '1156'}

[zRzRzRzRzRzRzR]

API在对接的时候是不做执行工具的部分的，这部分应该使用工程实现，可以提一个pr

[SongSongK]

API在对接的时候是不做执行工具的部分的，这部分应该使用工程实现，可以提一个pr

现有的api_server.py的接口不兼容openai_api的格式吗，我尝试工具调用没有按预期返回数据