THUNLP-MT / StableToolBench

A new tool learning benchmark aiming at well-balanced stability and reality, based on ToolBench.
https://zhichengg.github.io/stb.github.io/
Apache License 2.0
116 stars 15 forks source link

inference problem #15

Closed farawayxxx closed 1 month ago

farawayxxx commented 5 months ago

Thanks for your work!

When I tried to reproduce the inference, I encountered an issue. Is this normal?

openai.BadRequestError: Error code: 400 - {'error': {'message': "None is not of type 'array' - 'messages.6.tool_calls'", 'type': 'invalid_request_error', 'param': None, 'code': None}}

The last message sent to OpenAI was: {'content': "I have gathered information related to the 'Music' category from Vimeo. Here are some of the related tags for the 'Music' category: \n1. The Evolution of Pop Music\n2. Classical Music for Beginners\n3. The Rise of Electronic Dance Music\n4. Jazz: The Sound of Innovation\n5. Rock Legends: The Bands That Shaped a Generation\n\nAdditionally, the 'Music' category from Vimeo provides a comprehensive API designed to offer information on various music genres, artists, albums, tracks, metadata, lyrics, and streaming links for songs. It caters to developers, music enthusiasts, and industry professionals looking to integrate music data into their projects or applications.", 'role': 'assistant', 'tool_calls': None}

Here is the json_data: {'model': 'gpt-35-turbo-0125', 'messages': [{'role': 'system', 'content': 'You are AutoGPT, you can use many tools(functions) to do the following task.\nFirst I will give you the task description, and your task start.\nAt each step, you need to give your thought to analyze the status now and what to do next, with a function call to actually excute your step.\nAfter the call, you will get the call result, and you are now in a new state.\nThen you will analyze your status now, then decide what to do next...\nAfter many (Thought-call) pairs, you finally perform the task, then you can give your finial answer.\nRemember: \n1.the state change is irreversible, you can\'t go back to one of the former state, if you want to restart the task, say "I give up and restart".\n2.All the thought is short, at most in 5 sentence.\n3.You can do more then one trys, so if your plan is to continusly try some conditions, you can do one of the conditions per try.\nLet\'s Begin!\nTask description: You should use functions to help handle the real time user querys. Remember:\n1.ALWAYS call "Finish" function at the end of the task. And the final answer should contain enough information to show to the user,If you can\'t handle the task, or you find that function calls always fail(the function is not valid now), use function Finish->give_up_and_restart.\n2.Do not use origin tool names, use only subfunctions\' names.\nYou have access of the following tools:\n1.ytstream_download_youtube_videos: Download or stream YouTube Videos/MP4.\n2.vimeo: Most supportive community of creators and get high-quality tools for hosting, sharing, and streaming videos.\n'}, {'role': 'user', 'content': "\nI want to create a playlist of my favorite music videos from YouTube. Get the related tags for the 'Music' category from Vimeo to enhance the playlist. Also, provide me with information about the 'Music' category from Vimeo.\nBegin!\n"}, {'content': None, 'role': 'assistant', 'tool_calls': [{'id': 'call_ZS0f36QVhyGXhi4P2AdebPJw', 'function': {'arguments': '{"category":"Music","format":"json"}', 'name': 'getrelatedtags_for_vimeo'}, 'type': 'function'}]}, {'role': 'tool', 'name': 'getrelatedtags_for_vimeo', 'content': '{"error": "", "response": {"category": "Music", "format": "json", "page": 1, "per_page": 10, "results": [{"id": 1, "title": "The Evolution of Pop Music", "description": "An in-depth analysis of how pop music has evolved over the decades, from the 1950s to the present day.", "genre": "Pop"}, {"id": 2, "title": "Classical Music for Beginners", "description": "A beginner\'s guide to classical music, covering essential composers and their most iconic pieces.", "genre": "Classical"}, {"id": 3, "title": "The Rise of Electronic Dance Music", "description": "Exploring the origins and explosive growth of electronic dance music in the global scene.", "genre": "Electronic"}, {"id": 4, "title": "Jazz: The Sound of Innovation", "description": "Discover the rich history of jazz music and its influence on other music genres.", "genre": "Jazz"}, {"id": 5, "title": "Rock Legends: The Bands That Shaped a Generation", "description": "A retrospective look at the bands and artists who defined the rock genre and influenced countles...', 'tool_call_id': 'call_ZS0f36QVhyGXhi4P2AdebPJw'}, {'content': None, 'role': 'assistant', 'tool_calls': [{'id': 'call_K71rFylwj6PhjieSjsLJLWam', 'function': {'arguments': '{"category":"Music","format":"json"}', 'name': 'getcategoryinfo_for_vimeo'}, 'type': 'function'}]}, {'role': 'tool', 'name': 'getcategoryinfo_for_vimeo', 'content': '{"error": "", "response": "The API you\'ve queried is designed to provide information related to the \'Music\' category. Given the \'format\' parameter is set to \'json\', the response will be structured in JSON format, ensuring compatibility with a wide range of applications and services. This API can serve various purposes, such as retrieving details about music genres, artists, albums, or tracks. It can also be used to fetch metadata, lyrics, or even streaming links for songs. Depending on the specific endpoint called within this category, you might receive detailed descriptions of artists, including their biography, discography, and tour dates, or you might get structured data about albums, including track lists, release dates, and cover art. The API aims to be a comprehensive source for music-related data, catering to developers, music enthusiasts, and industry professionals seeking to integrate music information into their projects or applications."}', 'tool_call_id': 'call_K71rFylwj6PhjieSjsLJLWam'}, {'content': "I have gathered information related to the 'Music' category from Vimeo. Here are some of the related tags for the 'Music' category: \n1. The Evolution of Pop Music\n2. Classical Music for Beginners\n3. The Rise of Electronic Dance Music\n4. Jazz: The Sound of Innovation\n5. Rock Legends: The Bands That Shaped a Generation\n\nAdditionally, the 'Music' category from Vimeo provides a comprehensive API designed to offer information on various music genres, artists, albums, tracks, metadata, lyrics, and streaming links for songs. It caters to developers, music enthusiasts, and industry professionals looking to integrate music data into their projects or applications.", 'role': 'assistant', 'tool_calls': None}], 'max_tokens': 1024, 'frequency_penalty': 0, 'presence_penalty': 0, 'tools': [{'type': 'function', 'function': {'name': 'download_stream_for_ytstream_download_youtube_videos', 'description': 'This is the subfunction for tool "ytstream_download_youtube_videos", you can use this tool.The description of this function is: "Stream or download info."', 'parameters': {'type': 'object', 'properties': {'is_id': {'type': 'string', 'description': 'Youtube Video Id.', 'example_value': 'UxxajLWwzqY'}, 'cgeo': {'type': 'string', 'description': 'Country code in ISO 3166 format of the end user.'}}, 'required': ['', 'is_id'], 'optional': ['', 'cgeo']}}}, {'type': 'function', 'function': {'name': 'getrelatedtags_for_vimeo', 'description': 'This is the subfunction for tool "vimeo", you can use this tool.The description of this function is: "Get a list of related tags for a category."', 'parameters': {'type': 'object', 'properties': {'category': {'type': 'string', 'description': 'The Name of category'}, 'format': {'type': 'string', 'description': 'json or xml or php'}, 'page': {'type': 'integer', 'description': 'The page number to show'}, 'per_page': {'type': 'integer', 'description': 'Number of items to show on each page. Max 50.'}}, 'required': ['', 'category', 'format'], 'optional': ['', 'page', 'per_page']}}}, {'type': 'function', 'function': {'name': 'getcategoryinfo_for_vimeo', 'description': 'This is the subfunction for tool "vimeo", you can use this tool.', 'parameters': {'type': 'object', 'properties': {'category': {'type': 'string', 'description': 'The name of the category.'}, 'format': {'type': 'string', 'description': 'json or xml or php'}}, 'required': ['', 'category', 'format'], 'optional': ['']}}}, {'type': 'function', 'function': {'name': 'getchannelinfo_for_vimeo', 'description': 'This is the subfunction for tool "vimeo", you can use this tool.The description of this function is: "Get the information on a single channel."', 'parameters': {'type': 'object', 'properties': {'format': {'type': 'string', 'description': 'json/xml/php'}, 'channel_id': {'type': 'string', 'description': 'The numeric id of the channel or its url name.'}}, 'required': ['', 'format', 'channel_id'], 'optional': ['']}}}, {'type': 'function', 'function': {'name': 'Finish', 'description': 'If you believe that you have obtained a result that can answer the task, please call this function to provide the final answer. Alternatively, if you recognize that you are unable to proceed with the task in the current state, call this function to restart. Remember: you must ALWAYS call this function at the end of your attempt, and the only part that will be shown to the user is the final answer, so it should contain sufficient information.', 'parameters': {'type': 'object', 'properties': {'return_type': {'type': 'string', 'enum': ['give_answer', 'give_up_and_restart']}, 'final_answer': {'type': 'string', 'description': 'The final answer you want to give the user. You should have this field if "return_type"=="give_answer"'}}, 'required': ['return_type']}}}]}