OpenBMB / XAgent

An Autonomous LLM Agent for Complex Task Solving
https://blog.x-agent.net/blog/xagent/
Apache License 2.0
8.19k stars 845 forks source link

自定义添加的工具无法正确调用 #373

Open liyoung1992 opened 11 months ago

liyoung1992 commented 11 months ago

Issue Description / 问题描述

Please provide a detailed description of the error or issue you encountered. / 请详细描述您遇到的错误或问题。 定义 async def contour_detection(img_path: str = '', run_async:bool=False):后,上传图片,提问do contour detection for this image.You can't use the network, you can only use local registration tools for processing`,一直重复调用contour_detection,且参数没有匹配上。

Steps to Reproduce / 复现步骤

Please provide the specific steps to reproduce the error. / 请提供复现错误的具体步骤。 `from core.register import toolwrapper from core.exceptions import ToolExecutionError from config import CONFIG,logger import os import random import cv2 import numpy as np

@toolwrapper() async def contour_detection(img_path: str = '', run_async: bool=False): """ This is a new tool that do contour detection on the image.

Example:
```
in: contour_detection(img_path = "/work/test.png", run_async=True)
Out: {contour_detection_id: 1}# You can use this id to read the output and error later.
```

:param string img_path: The first parameter of the contour_detection function. It is a string.it is original image path of contour detection.
:param bool run_async: Whether to run the command asynchronously, default is False. If True, call this tool again with shell_id to get the final output and error.
:return: The output of the contour_detection function as a json string. contour_detection_result_id is get result id,you can use contour_detection_id to get detail result
:raises ToolExecutionError: If something goes wrong during the execution of the new_tool function.
"""
if os.path.exists(img_path) is False:
    raise ToolExecutionError({'Error': 'image is not exist!'})
    return
# Do something with param1 and param2
# If something goes wrong, raise a ToolExecutionError
# Return the output as a string
image = cv2.imread(img_path)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV)

contours, hierarchy = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

contour_image = np.zeros_like(image)

for contour in contours:
    cv2.drawContours(contour_image, [contour], -1, (0, 255, 0), 2)
random_id = random.randint(1000, 10000)
output_path = 'contour_image_' + str(random_id) + ".jpg"
cv2.imwrite(output_path, contour_image)
full_path = os.path.abspath(output_path)

logger.error("call contour_detection")
logger.error(img_path)
logger.error(run_async)
print("call contour_detection")

result = {
    'contour_detection_result_id': random_id
}
return str(result)`

Expected Behavior / 预期行为

Describe the behavior you expected to see. / 请描述您期望的正确行为。

Environment / 环境信息

Error Screenshots or Logs / 错误截图或日志

If possible, please provide relevant screenshots or logs of the error. / 如果可能,请提供相关的错误截图或日志文件。

using Tools:
Command Name:
"contour_detection"
Arguments:
{
img_path:"/work//test.png"
run_async:true
}
Execution Results:{"detail":{"Error":"image is not exist!"}}
Command Status:❌TOOL_CALL_FAILED

Additional Notes / 其他备注

If you have any additional information or notes, please add them here. / 如果有其他补充信息,请在此处添加。 step详细信息:

thoughts:
To select relevant features from an input image, we can use a variety of techniques such as edge detection or corner detection. These methods will help us identify key points within the image.
reasoning:
The goal is to implement a feature selection method that identifies representative regions in an image. Edge detection and corner detection are commonly used techniques because they involve finding areas with high gradient magnitudes or strong curvature, respectively. By applying these algorithms to our images, we should be able to obtain useful information about their structure.
plan:
Use the FileSystemEnv_search tool to find suitable feature selection methods for image processing
criticism:
This approach assumes that one of the available tools (FileSystemEnv_search) can provide functions specifically designed for performing edge detection or corner detection on images without using external resources.
ChamfersChen commented 9 months ago

请问,按照官方给的README,自定义tool脚本之后,要重新docker compose build,tool才会生效并写入prompt的function么?还是说有更快捷的方式?