PaddlePaddle / ERNIE-SDK

ERNIE Bot Agent is a Large Language Model (LLM) Agent Framework, powered by the advanced capabilities of ERNIE Bot and the platform resources of Baidu AI Studio.
http://ernie-bot-agent.readthedocs.io/
Apache License 2.0
324 stars 48 forks source link

function call出了正确结果了,但是最后一步模型回答的还是错误的答案。 #325

Closed illiterate closed 4 months ago

illiterate commented 4 months ago

按照官方的erniebot-agent/cookbook/local_tool.ipynb,改了一个天气查询的demo,明明function call出了正确的结果,但是最后一步模型回答的还是错误的答案。

from __future__ import annotations
import os
os.environ["EB_AGENT_LOGGING_LEVEL"] = "DEBUG"
os.environ['EB_AGENT_ACCESS_TOKEN'] = 'xxx'

import asyncio

from typing import Any, Dict, Type, List
from pydantic import Field
from erniebot_agent.tools.base import Tool
from erniebot_agent.tools.schema import ToolParameterView
from erniebot_agent.agents.function_agent import FunctionAgent
from erniebot_agent.chat_models import ERNIEBot
from erniebot_agent.memory import WholeMemory

class WeatherInput(ToolParameterView):
    location: str = Field(description="要查询天气的地点")

class WeatherOutput(ToolParameterView):
    result: str = Field(description="查询天气的结果")

class WeatherTool(Tool):
    description: str = "查询某地天气,返回结果"
    input_type: Type[ToolParameterView] = WeatherInput
    ouptut_type: Type[ToolParameterView] = WeatherOutput

    def __init__(self) -> None:
        super().__init__()

    async def __call__(self, location: str) -> Dict[str, Any]:
        return location + '多云,25-30度'

agent = FunctionAgent(ERNIEBot("ernie-3.5"), tools=[WeatherTool()], memory=WholeMemory())
result =  asyncio.run(agent.run("今天北京天气怎么样?"))
print(result)

结果: image

itagan commented 4 months ago

1,模型原因,比如幻觉;2,模型API处理策略?比如ernie默认联网,再次找了一遍,得到的网上真实结果。忽略了传递的上下文

itagan commented 4 months ago

研究了下,是你的工具函数有些问题。应该这样子: async def call(self, location: str) -> Dict[str, Any]:

return location + '多云,25-30度'

    return {"result": f"<{location}>多云,25-30度"}

毕竟WeatherOutput结构中定义了result。

itagan commented 4 months ago

3.5在这个时候也能回答正确,不会产生幻觉or调用网络工具解决。4.0原代码直接说明工具定义有问题

illiterate commented 4 months ago

已解决,感谢🙏

itagan commented 4 months ago

👌