建议增加openai的function call特性

[jinghai]

https://platform.openai.com/docs/guides/gpt/function-calling

[xusenlinzy]

已经添加了通义千问模型的 function call 支持，使用方式参考 https://github.com/xusenlinzy/api-for-open-llm/blob/master/examples/quad_calculator.py

[yhfgyyf]

已经添加了通义千问模型的 function call 支持，使用方式参考 https://github.com/xusenlinzy/api-for-open-llm/blob/master/examples/quad_calculator.py

能否修改下route.py中function call的代码，符合openai api的接口，方便langchain的agent调用？

[zhengxiang5965]

@xusenlinzy 使用OpenAIFunctionAgent 调用，会报错

KeyError: 'name_for_model' INFO: 172.16.30.94:64022 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi result = await app( # type: ignore[func-returns-value] File "/usr/local/lib/python3.8/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in call return await self.app(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/fastapi/applications.py", line 276, in call await super().call(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/cors.py", line 84, in call await self.app(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/usr/local/lib/python3.8/dist-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in call raise e File "/usr/local/lib/python3.8/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in call await self.app(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 718, in call await route.handle(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.8/dist-packages/starlette/routing.py", line 66, in app response = await func(request) File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/usr/local/lib/python3.8/dist-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(**values) File "/workspace/./api/router.py", line 270, in create_chat_completion gen_params = get_gen_params( File "/workspace/./api/router.py", line 115, in get_gen_params messages = get_qwen_react_prompt(messages, functions, function_call) File "/workspace/./api/react_prompt.py", line 35, in get_qwen_react_prompt name_for_model=info["name_for_model"], KeyError: 'name_for_model'

[zhengxiang5965]

2023-08-08 15:27:25 [llm/start] [1:chain:AgentExecutor > 2:llm:ChatOpenAI] Entering LLM run with input: { 2023-08-08 15:27:25 "messages": [ 2023-08-08 15:27:25 [ 2023-08-08 15:27:25 { 2023-08-08 15:27:25 "lc": 1, 2023-08-08 15:27:25 "type": "constructor", 2023-08-08 15:27:25 "id": [ 2023-08-08 15:27:25 "langchain", 2023-08-08 15:27:25 "schema", 2023-08-08 15:27:25 "SystemMessage" 2023-08-08 15:27:25 ], 2023-08-08 15:27:25 "kwargs": { 2023-08-08 15:27:25 "content": "", 2023-08-08 15:27:25 "additional_kwargs": {} 2023-08-08 15:27:25 } 2023-08-08 15:27:25 }, 2023-08-08 15:27:25 { 2023-08-08 15:27:25 "lc": 1, 2023-08-08 15:27:25 "type": "constructor", 2023-08-08 15:27:25 "id": [ 2023-08-08 15:27:25 "langchain", 2023-08-08 15:27:25 "schema", 2023-08-08 15:27:25 "HumanMessage" 2023-08-08 15:27:25 ], 2023-08-08 15:27:25 "kwargs": { 2023-08-08 15:27:25 "content": "将3号设备的状态通知全公司", 2023-08-08 15:27:25 "additional_kwargs": {} 2023-08-08 15:27:25 } 2023-08-08 15:27:25 } 2023-08-08 15:27:25 ] 2023-08-08 15:27:25 ] 2023-08-08 15:27:25 }

[xusenlinzy]

拉取最新的代码（效果不稳定，可能是langchain里面有很多默认的prompt）

from langchain import LLMMathChain
from langchain.agents import AgentType
from langchain.agents import initialize_agent, Tool
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(
    openai_api_base="http://192.168.0.53:7891/v1",
    openai_api_key="xxx",
    model="qwen",
    temperature=0,
)
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)

tools = [
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for when you need to answer questions about math"
    ),
]

agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
agent.run("312+512等于多少？")

结果为

[wuzechuan]

调用多个工具的时候，有时候qwen会拒绝使用工具或者不用我提供的工具，可以提供langchain 的OPENAI_MULTI_FUNCTIONS 的案例吗

[yhfgyyf]

拉取最新的代码（效果不稳定，可能是langchain里面有很多默认的prompt）

from langchain import LLMMathChain
from langchain.agents import AgentType
from langchain.agents import initialize_agent, Tool
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(
    openai_api_base="http://192.168.0.53:7891/v1",
    openai_api_key="xxx",
    model="qwen",
    temperature=0,
)
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)

tools = [
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for when you need to answer questions about math"
    ),
]

agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
agent.run("312+512等于多少？")

结果为

好像不能理解langchain的prompt，日志上只有prompt，有没有回答的日志看下？

[yhfgyyf]

langchain中有个利用function call生成python shell的prompt，好像不支持啊

[zhengxiang5965]

stream模式下，function call 不兼容 这是非stream模式下响应体 这是stream模式下

stream没有对function call做适配,没有function_call属性

@xusenlinzy

[uulichen]

function call没有生效。尝试了用langchain或者原生的openai API 都不成功。 返回结果中function_call为空。

debug log ： { 'model': 'qwen', 'prompt': [ChatMessage(role = 'user', content = 'Answer the following questions as best you can. You have access to the following tools:\n\nis_blocked: Call this tool to interact with the is_blocked API. What is the is_blocked API useful for? determine if the provided text is prevent crawling and suggest proper action Parameters: {"type": "object", "properties": {"thoughts": {"type": "object", "properties": {"reasoning": {"type": "string", "description": "Provide reasons whether this text is related to security protection."}, "action": {"type": "string", "description": "continue or terminate"}}, "required": ["action", "reasoning"]}}, "required": ["thoughts"]} Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [is_blocked]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: I have crawled a piece of text from the internet, but due to the website\'s security measures, the text may not be the article I need. \nInstead, it could be a message displayed by the website to prevent crawling or blocking access, such as \'Access temporarily blocked for security reasons\' or \'Security violation error.\' \nNow I need you to determine if the provided text falls into this category and suggest proper action.\n\nThere are two actions you need to choose from:\n"continue": if the text is not related to security measures and crawling prevention.\n"terminate": if the text is related to security measures and crawling prevention.\nIf you are unable to make a judgment, please choose to continue.\n\nbelow is the text for you to review:\n开发者祭出离谱新操作：苹果Vision Pro虚拟机运行Windows XP成可能 - Windows XP - cnBeta.COM cnBeta.COM.TW_中文业界资讯站 首页 影视 音乐 游戏 动漫 趣闻 科学 软件 主题 硬件 Apple Google iPhone 科学探索 人物 手机 游戏 视点·观察 阿里云 微软 通信技术 Android 软件和应用 SONY 索尼 the United States美国 Apple iPad Windows Phone Intel英特尔 腾讯 HTC 安全 Mozilla FireFox 小米科技 百度 通信运营商 媒体播放器 / 视频网站 BlackBerry 黑莓 网络应用 / Web Apps Google Chrome Microsoft XBOX 硬件 - 上网本 / 平板电脑 / 超极本 电子商务 - B2C / B2B Lenovo 联想 LG Yahoo! 雅虎 AMD SNS 社交网络 诺基亚 Windows 7 Huawei 华为 中国移动 中国联通 中国电信 更多... 排行 搜索 开发者祭出离谱新操作：苹果Vision Pro虚拟机运行Windows XP成可能 2023年08月10日 10:29 792 次阅读 稿源： 快科技 0 条评论 近日，虚拟机项目UTM的开发者，在推特分享了一段视频， 展示了在VisionOS官方模拟器中，成功启动Windows XP虚拟机的过程。 这意味着，在Vision Pro中，使用WinXP或是后续的Windows操作系统，已经成为可能。 不过，根据开发者介绍，虽然目前已经成功在VisionOS中运行WinXP，但他们还没有实现输入设备的支持。 也就是说， 目前在VisionOS中启动WinXP后，用户暂时无法与之进行任何交互。 这一功能将在UTM后续的开发中实现支持。 UTM是一个已经适配iOS、 iPad OS与macOS的虚拟机软件，它能够帮助用户在 iPhone \n', name = None, functions = None, function_call = None)], 'temperature': 0.0, 'top_p': 1.0, 'max_new_tokens': 1024, 'echo': False, 'stream': False, 'with_function_call': True, 'stop': ['<|im_end|>', 'Observation:'] }

[xusenlinzy]

@uulichen 目前function call功能只能支持简单的工具调用，和langchain的适配还不是很好，你可以先参考这里的用法：https://github.com/xusenlinzy/api-for-open-llm/blob/master/examples/qwen-7b-chat/react_demo.py