OpenBMB / XAgent

An Autonomous LLM Agent for Complex Task Solving
https://blog.x-agent.net/blog/xagent/
Apache License 2.0
7.82k stars 794 forks source link

执行Task报错 XAgentServer.exts.exception_ext.XAgentRunningError: 'NoneType' object is not subscriptable Error in task_handler of 5bee2b1b3c7b40518d847e49b98669eb: 'NoneType' object is not subscriptable #371

Open itq5 opened 5 months ago

itq5 commented 5 months ago

Issue Description / 问题描述

使用One-api项目部署的api,在执行Task时,能看到one-api有做出相应和回复,但是XAgent执行报错了。

Steps to Reproduce / 复现步骤

Expected Behavior / 预期行为

Environment / 环境信息

Error Screenshots or Logs / 错误截图或日志

XAgent日志:

root@27910cc30d1d:/app# python run.py --task "test" --config-file "assets/config.yml"
XAgentServer is running on cmd mode  
XAgentServer log:  XAgentServer/localstorage/interact_records/2024-01-04/5bee2b1b3c7b40518d847e49b98669eb
  init interaction: 5bee2b1b3c7b40518d847e49b98669eb
  Register logger into interaction of 5bee2b1b3c7b40518d847e49b98669eb, done!
  Register db into interaction of 5bee2b1b3c7b40518d847e49b98669eb, done!
  Register logger into XAgentServer of 5bee2b1b3c7b40518d847e49b98669eb, done!
  Start a new thread to run interaction of 5bee2b1b3c7b40518d847e49b98669eb, done!
---config file---
assets/config.yml
---args---
{'task': 'test', 'mode': 'auto', 'quiet': False, 'enable_ask_human_for_help': False, 'config_file': 'assets/config.yml'}
  server is running, the start query is test
-=-=-=-=-=-=-=Recorder Start-=-=-=-=-=-=-=
  Current: None Node: query 0
-=-=-=-=-=-=-=Data -=-=-=-=-=-=-=
  {
    "task": "test",
    "role_name": "Assistant",
    "plan": []
}
-=-=-=-=-=-=-=Recorder End-=-=-=-=-=-=-=  
-=-=-=-=-=-=-=Recorder Start-=-=-=-=-=-=-=
  Current: None Node: config 0
-=-=-=-=-=-=-=Data -=-=-=-=-=-=-=
  {
    "api_keys": {
        "gpt-3.5-turbo-16k": [
            {
                "api_key": "**",
                "api_base": "https://api.***.com/v1",
                "model": "gpt-3.5-turbo-16k"
            }
        ],
        "gpt-4": [
            {
                "api_key": "**",
                "api_base": "https://api.***.com/v1",
                "model": "gpt-4"
            }
        ],
        "gpt-4-32k": [
            {
                "api_key": "**",
                "api_base": "https://api.***.com/v1",
                "model": "gpt-4-32k"
            }
        ]
    },
    "default_request_type": "openai",
    "default_completion_kwargs": {
        "model": "gpt-3.5-turbo-16k",
        "temperature": 0.2,
        "request_timeout": 60
    },
    "enable_summary": true,
    "summary": {
        "single_action_max_length": 2048,
        "max_return_length": 12384
    },
    "use_selfhost_toolserver": true,
    "selfhost_toolserver_url": "http://ToolServerManager:8080",
    "max_retry_times": 5,
    "max_subtask_chain_length": 15,
    "max_plan_refine_chain_length": 3,
    "max_plan_tree_depth": 3,
    "max_plan_tree_width": 5,
    "max_plan_length": 8192,
    "rapidapi_retrieve_tool_count": 0,
    "enable_ask_human_for_help": false,
    "tool_blacklist": [
        "FileSystemEnv_print_filesys_struture"
    ],
    "record_dir": null,
    "experiment": {
        "redo_action": false
    },
    "task": "test",
    "mode": "auto",
    "quiet": false,
    "config_file": "assets/config.yml"
}
-=-=-=-=-=-=-=Recorder End-=-=-=-=-=-=-=  
  register tool server interface
  lazy init tool server interface
ToolServer connected in  http://ToolServerManager:8080
  register function handler
  register working memory function
  register agent dispatcher
Constructing an AgentDispatcher:  XAgentDispatcher
  build all components, done!
-=-=-=-=-=-=-=Recorder Start-=-=-=-=-=-=-=
  Current: None Node: tool_server_pair 0
-=-=-=-=-=-=-=Data -=-=-=-=-=-=-=
  {
    "url": "get_available_tools",
    "payload": {},
    "response_status_code": 200,
    "tool_output": {
        "available_envs": [
            {
                "name": "FileSystemEnv",
                "description": "Provide a file system operation environment for Agent.\n    ",
                "total_tools": 3,
                "tools": [
                    "print_filesys_struture",
                    "read_from_file",
                    "write_to_file"
                ]
            },
            {
                "name": "WebEnv",
                "description": "Web Environment providing web interface and browsering.\n    ",
                "total_tools": 2,
                "tools": [
                    "browse_website",
                    "search_and_browse"
                ]
            },
            {
                "name": "PythonNotebook",
                "description": "Python Notebook Environment. Provide a notebook interface to run python code.",
                "total_tools": 2,
                "tools": [
                    "execute_cell",
                    "print_notebook"
                ]
            }
        ],
        "available_tools": [
            "FileSystemEnv_print_filesys_struture",
            "FileSystemEnv_read_from_file",
            "FileSystemEnv_write_to_file",
            "WebEnv_browse_website",
            "WebEnv_search_and_browse",
            "PythonNotebook_execute_cell",
            "PythonNotebook_print_notebook",
            "shell_command_executor"
        ],
        "tools_json": [
            {
                "name": "FileSystemEnv_print_filesys_struture",
                "description": "Return a tree-like structure for all files and folders in the workspace. Use this tool if you are not sure what files are in the workspace.\nThis function recursively walks through all the directories in the workspace\nand return them in a tree-like structure, \ndisplaying all the files under each directory.\n\nExample:\n```\n- root/\n    - sub_directory1/\n        - file1.txt\n        - file2.txt\n    - sub_directory2/\n        - file3.txt\n```",
                "parameters": {
                    "type": "object",
                    "properties": {},
                    "required": []
                }
            },
            {
                "name": "FileSystemEnv_read_from_file",
                "description": "Open and read the textual file content in the workspace, you will see the content of the target file.\nDon't use this if the give `filepath` is writen or modified before, the content in `filepath` should be already returned.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "filepath": {
                            "type": "string",
                            "description": "The path to the file to be opened, always use relative path to the workspace root."
                        },
                        "line_number": {
                            "type": "integer",
                            "description": "The starting line number of the content to be opened. Defaults to 1."
                        }
                    },
                    "required": [
                        "filepath"
                    ]
                }
            },
            {
                "name": "FileSystemEnv_write_to_file",
                "description": "Write or modify the textual file lines based on `content` provided. \nReturn updated content of the file after modification so no further need to call `read_from_file` for this file. Create file if not exists.\n\nExample:\n```\nIn[0]: write_to_file('test.txt', 'Hello World!\\nA new line!')\nOut[0]: '1: Hello World!\\n2: A new line!'\nIn[1]: write_to_file('test.txt', 'Hello World 1!', 2)\nOut[1]: '1: Hello World!\\n2: Hello World 1!\\n3: A new line!'\nIn[2]: write_to_file('test.txt', 'Hello World 2!', 2, overwrite=True)\nOut[2]: '1: Hello World!\\n2: Hello World 2!\\n3: A new line!'\n```",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "filepath": {
                            "type": "string",
                            "description": "The path to the file to be modified, always use relative path to the workspace root."
                        },
                        "truncating": {
                            "type": "boolean",
                            "description": "If `True`, the file will be truncated before writing, else will read current content before writing. Defaults to `False`."
                        },
                        "line_number": {
                            "type": "integer",
                            "description": "The start line to modified file. Defaults to `None`, which means insert the new content at the end of the file. So do not provide this if you want to append the new content to the file."
                        },
                        "overwrite": {
                            "type": "boolean",
                            "description": "If `True`, the new content will overwrite content started from `line_number` line. Defaults to `False`, which insert the new content at the `line_number` line."
                        },
                        "content": {
                            "type": "string",
                            "description": "The new content to be replaced with the old content."
                        }
                    },
                    "required": [
                        "filepath",
                        "content"
                    ]
                }
            },
            {
                "name": "WebEnv_browse_website",
                "description": "Give a http or https url to browse a website and return the summarize text. Note some websites may not be accessable due to network error. This tool only return the content of give url and cannot provide any information need interaction with the website.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "url": {
                            "type": "string",
                            "description": "The realworld Uniform Resource Locator (web address) to scrape text from. Never provide something like \"<URL of the second news article>\", give real url!!! Example: 'https://www.deepmind.com/'"
                        },
                        "goals_to_browse": {
                            "type": "string",
                            "description": "The goals for browse the given `url` (e.g. what you want to find on webpage.). If you need more details, request it in here."
                        }
                    },
                    "required": [
                        "url",
                        "goals_to_browse"
                    ]
                }
            },
            {
                "name": "WebEnv_search_and_browse",
                "description": "Search with search tools and browse the website returned by search. Note some websites may not be accessable due to network error.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "search_query": {
                            "type": "string",
                            "description": "The search query."
                        },
                        "goals_to_browse": {
                            "type": "string",
                            "description": "What's you want to find on the website returned by search. If you need more details, request it in here. Examples: 'What is latest news about deepmind?', 'What is the main idea of this article?'"
                        },
                        "region": {
                            "type": "string",
                            "description": "The region code of the search, default to `en-US`. Available regions: `en-US`, `zh-CN`, `ja-JP`, `de-DE`, `fr-FR`, `en-GB`."
                        }
                    },
                    "required": [
                        "search_query",
                        "goals_to_browse"
                    ]
                }
            },
            {
                "name": "PythonNotebook_execute_cell",
                "description": "Create or replace a notebook cell and execute it, return the output.\nUse this tool to test your idea quickly. Carefully examine the output to make sure it is what you want.\n\nExample:\n```\nIn[0]: code='print(\"hello world\")' # This will create a new cell and execute it.\nOut[0]: ['cell_index: 0', 'hello world']\nIn[1]: code='print(\"hello world\")',cell_index=0 # This will overwrite the first cell and execute it.\nIn[2]: code='print(\"hello world\")',cell_index=-1 # This will overwrite the last cell and execute it.\n```",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "code": {
                            "type": "string",
                            "description": "python code to be executed, make sure it is valid python code with right format. don't provide shell command that started with '!' here."
                        },
                        "cell_index": {
                            "type": "integer",
                            "description": "the index of the cell to be insert and overwrite `code`, default to `None`, which means append new cell."
                        },
                        "reset": {
                            "type": "boolean",
                            "description": "whether to reset the kernel before executing the code. Default to `False`."
                        }
                    },
                    "required": [
                        "code"
                    ]
                }
            },
            {
                "name": "PythonNotebook_print_notebook",
                "description": "print all notebook cells' content and output.",
                "parameters": {
                    "type": "object",
                    "properties": {},
                    "required": []
                }
            },
            {
                "name": "shell_command_executor",
                "description": "The shell tool that execute shell command in root privilege, return the output and error. \nYou can use this tool to install packages, download files, run programs, etc.\nSet run_async=True to run the command in a new thread and return instantly if your command is time costly like install packages, host services. \nExample:\n```\nIn: shell_command_executor(command='echo \"hello world\"')\nOut: \"hello world\"\nIn: shell_command_executor(command='sleep 10', run_async=True)\nOut: {'shell_id': 0} # You can use this id to read the output and error later.\nIn: shell_command_executor(shell_id=0, kill=True)\nOut: \"\" # The shell 0 will be killed.\n```",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "command": {
                            "type": "string",
                            "description": "The shell command to be executed, must avoid command requiring additional user input. Default is empty string."
                        },
                        "run_async": {
                            "type": "boolean",
                            "description": "Whether to run the command asynchronously, default is False. If True, call this tool again with shell_id to get the final output and error."
                        },
                        "shell_id": {
                            "type": "integer",
                            "description": "The id of shell to execute command, default is None, which means running in a new shell. Change this to execute command in the same shell."
                        },
                        "kill": {
                            "type": "boolean",
                            "description": "If True, kill the shell which runs the command after execution. Default is False. Don't use any other kill command!"
                        }
                    },
                    "required": []
                }
            }
        ]
    }
}
-=-=-=-=-=-=-=Recorder End-=-=-=-=-=-=-=  
  {
  "api_keys": {
    "gpt-3.5-turbo-16k": [
      {
        "api_key": "**",
        "api_base": "https://api.itq5.com/v1",
        "model": "gpt-3.5-turbo-16k"
      }
    ],
    "gpt-4": [
      {
        "api_key": "**",
        "api_base": "https://api.itq5.com",
        "model": "gpt-4"
      }
    ],
    "gpt-4-32k": [
      {
        "api_key": "**",
        "api_base": "https://api.itq5.com",
        "model": "gpt-4-32k"
      }
    ]
  },
  "default_request_type": "openai",
  "default_completion_kwargs": {
    "model": "gpt-3.5-turbo-16k",
    "temperature": 0.2,
    "request_timeout": 60
  },
  "enable_summary": true,
  "summary": {
    "single_action_max_length": 2048,
    "max_return_length": 12384
  },
  "use_selfhost_toolserver": true,
  "selfhost_toolserver_url": "http://ToolServerManager:8080",
  "max_retry_times": 5,
  "max_subtask_chain_length": 15,
  "max_plan_refine_chain_length": 3,
  "max_plan_tree_depth": 3,
  "max_plan_tree_width": 5,
  "max_plan_length": 8192,
  "rapidapi_retrieve_tool_count": 0,
  "enable_ask_human_for_help": false,
  "tool_blacklist": [
    "FileSystemEnv_print_filesys_struture"
  ],
  "record_dir": null,
  "experiment": {
    "redo_action": false
  },
  "task": "test",
  "mode": "auto",
  "quiet": false,
  "config_file": "assets/config.yml"
}
Human-In-The-Loop  False
  Start outer loop async
-=-=-=-=-=-=-= BEGIN QUERY SOVLING -=-=-=-=-=-=-=  
Role  Assistant
Task  test
-=-=-=-=-=-=-= GENERATE INITIAL_PLAN -=-=-=-=-=-=-= 
Constructing an Agent:  PlanGenerateAgent
  chatcompletion: using gpt-3.5-turbo-16k
Traceback (most recent call last):
  File "/app/XAgentServer/server.py", line 104, in interact
    task_handler.outer_loop()
  File "/app/XAgent/workflow/task_handler.py", line 82, in outer_loop
    self.plan_agent.initial_plan_generation(
  File "/app/XAgent/workflow/plan_exec.py", line 164, in initial_plan_generation
    new_message , _ = agent.parse(
  File "/app/XAgent/agent/plan_generate_agent/agent.py", line 49, in parse
    return self.generate(
  File "/app/XAgent/agent/base_agent.py", line 121, in generate
    response = objgenerator.chatcompletion(
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/app/XAgent/ai_functions/request/obj_generator.py", line 72, in chatcompletion
    response = self.function_call_refine(kwargs,response)
  File "/app/XAgent/ai_functions/request/obj_generator.py", line 194, in function_call_refine
    function_schema = list(filter(lambda x: x['name'] == response['choices'][0]['message']['function_call']['name'],req_kwargs['functions']))
  File "/app/XAgent/ai_functions/request/obj_generator.py", line 194, in <lambda>
    function_schema = list(filter(lambda x: x['name'] == response['choices'][0]['message']['function_call']['name'],req_kwargs['functions']))
TypeError: 'NoneType' object is not subscriptable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/command.py", line 273, in task_handler
    server.interact(interaction=interaction)
  File "/app/XAgentServer/server.py", line 106, in interact
    raise XAgentRunningError(str(e)) from e
XAgentServer.exts.exception_ext.XAgentRunningError: 'NoneType' object is not subscriptable
Error in task_handler of 5bee2b1b3c7b40518d847e49b98669eb: 'NoneType' object is not subscriptable  
root@27910cc30d1d:/app# 

One-API日志:

2024-01-04T03:06:48.671271529Z [INFO] 2024/01/04 - 11:06:48 | 20240104110648662797624PiuLMCxV | user 1 has enough quota 80338039, trusted and no need to pre-consume 
2024-01-04T03:06:56.450649057Z [GIN] 2024/01/04 - 11:06:56 | 20240104110648662797624PiuLMCxV | 200 |  7.787541511s | 155.248.170.111 |    POST /v1/chat/completions
2024-01-04T03:06:56.459031561Z [INFO] 2024/01/04 - 11:06:56 | 20240104110648662797624PiuLMCxV | record consume log: userId=1, 用户调用前余额=80338039, channelId=33, promptTokens=699, completionTokens=478, modelName=gpt-3.5-turbo-16k, tokenName=api, quota=2482, content=模型倍率 1.50,分组倍率 1.00,用时 8秒 

assets/config.yml 配置:

root@27910cc30d1d:/app# cat assets/config.yml 
api_keys:
  gpt-3.5-turbo-16k:
    - api_key: sk-51OqeowR**********************A0127aAf9a5bC5D1
      api_base: https://api.***.com/v1
      model: gpt-3.5-turbo-16k
  gpt-4:    
    - api_key: sk-51Oqeow***********************************9a5bC5D1
      api_base: https://api.***.com/v1
      model: gpt-4
  gpt-4-32k:
    - api_key: sk-51O***********************************9a5bC5D1
      api_base: https://api.***.com/v1
      model: gpt-4-32k

default_request_type: openai # or xagent
default_completion_kwargs:
  model: gpt-3.5-turbo-16k
  temperature: 0.2
  request_timeout: 60

enable_summary: true
summary:
  single_action_max_length: 2048
  max_return_length: 12384

use_selfhost_toolserver: true
selfhost_toolserver_url: http://localhost:8080

max_retry_times: 5
max_subtask_chain_length: 15
max_plan_refine_chain_length: 3
max_plan_tree_depth: 3
max_plan_tree_width: 5
max_plan_length: 8192

rapidapi_retrieve_tool_count: 0

enable_ask_human_for_help: False

tool_blacklist:
  - FileSystemEnv_print_filesys_struture

record_dir:

experiment:
  redo_action: false`

Additional Notes / 其他备注

帮我检查下XAgent如何修改进行适配

2252031668 commented 5 months ago

我使用了GPT4&3.5转发的API ,也出现了一模一样的问题,

itq5 commented 5 months ago

你的问题解决了吗?

我使用了GPT4&3.5转发的API ,也出现了一模一样的问题,

leap233 commented 5 months ago

请问解决了吗

pluswave commented 4 months ago

我本地打个补丁,感觉可能有点用。

--- a/XAgent/ai_functions/request/obj_generator.py
+++ b/XAgent/ai_functions/request/obj_generator.py
@@ -189,6 +189,8 @@ class OBJGenerator:
         if 'function_call' not in response['choices'][0]['message']:
             logger.typewriter_log("FunctionCallSchemaError: No function call found in the response", Fore.RED)
             raise FunctionCallSchemaError(f"No function call found in the response: {response['choices'][0]['message']} ")
+        if  not response['choices'][0]['message']['function_call']:
+            return response

         # verify the schema of the function call if exists
         function_schema = list(filter(lambda x: x['name'] == response['choices'][0]['message']['function_call']['name'],req_kwargs['functions']))
pluswave commented 4 months ago

感觉还是不行,应该是代码逻辑有问题。 function_call = None 传下去之后,发到 open_ai 的 API 会

{ ... "function_call": null }

从openai 文档看,这样调用是非法的。很可能得到的结果是 和

{ ... "function_call": "none" }

相同,而不是 "auto" , 而 "auto" 应该是预期的值。

pluswave commented 4 months ago

我发现了这个问题的根本原因了。 XAgent 通过 在 functions 里面提供两个函数

subtask_submit

subtask_handle

来让LLM选择一条,而subtask_handle 的内容需要调用 System prompt的Tools列表中的一项。 LLM 并没有按照调用预期,通过 function的返回来填充内容,反而直接把 tool的调用嵌入到了文本输出里面了。

> console.log(x.output.choices[0].message.content)
Given the information we have gathered and the preliminary structure of the "integration_technology_draft.md" file, the next step is to fill in each section with the summarized information. The information collected about NiceGUI, LangChain, OpenAI, Zhipuai, and Baichuanai provides a solid foundation for writing comprehensive documentation. This documentation will detail the key features, integration potentials, and challenges of integrating these technologies.

We begin by prioritizing the completion of the first section on NiceGUI, following the established outline. This step is logical as it allows for a focused and cohesive compilation of information. After completing the NiceGUI section, we will continue to fill in the rest of the sections systematically. This approach ensures efficiency and thoroughness, adhering to the task's objectives.

The critical information on NiceGUI extracted from the search results highlights its user-friendly interface, efficient connection management, automation in reconnection, and its suitability for tasks requiring continuous interaction. This information directly maps onto the designated subsections of key features, integration potential, and challenges.

Given the amount of text to be generated and the need for precise formatting, it is appropriate to utilize the FileSystemEnv to write to the document directly.

**Action Plan**: Populate the NiceGUI section in the "integration_technology_draft.md" file with the following points:
- Key Features: User-friendly interface, efficient connection management, reconnection automation, suitable for tasks requiring continuous interaction.
- Integration Potential: High compatibility with devices like webcams and GPIO pins in IoT setups, streamlines code management.
- Challenges: Potential challenges include ensuring stability in varied application scenarios and managing real-time interaction complexities.

**Tool Call:**

```json
{
  "tool_name": "FileSystemEnv_write_to_file",
  "tool_input": {
    "filepath": "integration_technology_draft.md",
    "content": "\n### NiceGUI\n- **Key Features:** User-friendly interface, efficient connection management, automation in reconnection, and suitability for tasks requiring continuous interaction. Compatible with peripheral devices and streamlines code management across applications.\n- **Integration Potential:** High due to its ability to enhance user experience with a simplified interface and efficient management of connected devices and real-time data streams in IoT and smart applications.\n- **Challenges:** Ensuring application stability across various use cases and managing the complexities of real-time data interaction and user-input processing.",
    "line_number": 4,
    "overwrite": true
  }
}

This action aims to incrementally complete the documentation by elaborating on NiceGUI, setting a precedent for detailing the other technologies in subsequent steps.

console.log(x.output.choices[0].message.function_call) null

不知为何这么设计,感觉把 tools调用放到 function 列表(或者 tools列表)里面,让LLM多选1,而不是现在的两层可能会好些。

关于 OpenAI的API , 发现一个问题

提供一组 functions / tools , "auto" 模式下,API的返回不能确保一定会选择一个 function / tool 。

换句话说,OpenAI提供三种函数/工具调用模式

但是缺少一种模式:

而XAgent提示词设计似乎就认为 LLM 的输出会使用这个缺失的模式。