xlang-ai / OpenAgents

[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild
https://arxiv.org/abs/2310.10634
Apache License 2.0
3.9k stars 425 forks source link

Tool selection doesn't help and agent is not adopting the tool for execution #67

Closed Jeffwan closed 9 months ago

Jeffwan commented 10 months ago

I feel it's really hard to reproduce the use cases here. https://docs.xlang.ai/use-cases/data-agent

kaggle tool is selected based on the output but not executed.. I never get lucky to get a result like what website shows (multiple kaggle dataset and I can click download to interact with)

image

openagents-backend-1   | 2023-11-03 22:10:53 | DEBUG - DefaultUser++65456f07e8aadbb3bc46a5d6->/chat New human message:{'message_type': 'human_message', 'message_content': 'Can you help me find datasets on Kaggle that specially focus on NBA play performance statistics over multiple seasons?', 'message_id': 37, 'parent_message_id': -1}
openagents-backend-1   | 2023-11-03 22:10:53 | DEBUG - DefaultUser++65456f07e8aadbb3bc46a5d6->/chat New ai message:{'message_type': 'ai_message', 'message_content': 'Sure, I can help you with that. Let me use the KaggleDataLoader tool to search for datasets related to NBA play performance statistics over multiple seasons.', 'message_id': 38, 'parent_message_id': 37}

openagents-backend-1   | 38.31.166.90 - - [03/Nov/2023 22:10:53] "OPTIONS /api/recommend HTTP/1.1" 200 -
openagents-backend-1   | 2023-11-03 22:10:54 | DEBUG - ++-> get_activated_message_list:[]
Timothyxxx commented 10 months ago

Thanks for your question! Could you try again with the GPT-4 endpoint? Just as I mentioned in https://github.com/xlang-ai/OpenAgents/issues/66#issuecomment-1793332707 , today's system with prompting mechanism rely on powerful LLMs to drive the cases

Timothyxxx commented 10 months ago

There are actually other ways to make it better, instead of prompting, check our tech report here.