geekan / MetaGPT

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming
https://deepwisdom.ai/
MIT License
41.53k stars 4.95k forks source link

Running "researcher" example fails when using gemini-pro model. #1050

Open vmsysadm opened 3 months ago

vmsysadm commented 3 months ago

Bug description Running "researcher" example fails when using gemini-pro model. It appears that the first step where David(Researcher) is passing the user query to CollectLinks(David) is not passing correct data. I'm not sure how to provide additional move verbose logging but will be happy to if someone would let me know.

Environment information System version - Windows 11 Python version - Python 3.10.9 LLM type and model - Gemini

Installation method: "development mode" git clone https://github.com/geekan/MetaGPT.git

config2.example.yaml: llm: api_type: 'gemini' api_key: 'XXXXXXXXXXXXXXXXXXXXXXXXXXX' model: 'gemini-pro'

search: api_type: "google" api_key: "XXXXXXXXXXXXXXXXXXXXXXXXXXX" cse_id: "XXXXXXXXXXXXXX"

(metagpt) PS C:\Users\user\code\MetaGPT> python -m metagpt.roles.researcher "meat vs icecream" 2024-03-19 13:08:32.406 | INFO | metagpt.const:get_metagpt_package_root:29 - Package root set to C:\Users\user\code\MetaGPT 2024-03-19 13:08:37.393 | INFO | main:_act:56 - David(Researcher): to do CollectLinks(David) ["Computer Vision", "Object Detection"] 2024-03-19 13:08:38.442 | INFO | metagpt.utils.cost_manager:update_cost:57 - Total running cost: $0.000 | Max budget: $10.000 | Current cost: $0.000, prompt_tokens: 37, completion_tokens: 9 Warning: model not found. Using cl100k_base encoding. Warning: model not found. Using cl100k_base encoding. ["What are the applications of computer vision in the healthcare industry?", "How can object detection be used to improve safety in autonomous vehicles?", "What are the challenges and limitations of computer vision in real-world applications?", "How is computer vision being used to advance scientific research?"] 2024-03-19 13:08:42.291 | INFO | metagpt.utils.cost_manager:update_cost:57 - Total running cost: $0.000 | Max budget: $10.000 | Current cost: $0.000, prompt_tokens: 1251, completion_tokens: 57 [ ...snip...]

PS C:\Users\user\code\MetaGPT\logs> less .\20240319.txt 2024-03-19 13:08:37.393 | DEBUG | metagpt.roles.role:_observe:431 - David(Researcher) observed: ['user: meat vs icecream...'] 2024-03-19 13:08:37.393 | DEBUG | metagpt.roles.role:_set_state:325 - actions=[CollectLinks, WebBrowseAndSummarize, ConductResearch], state=0 2024-03-19 13:08:37.393 | INFO | main:_act:56 - David(Researcher): to do CollectLinks(David) 2024-03-19 13:08:37.393 | DEBUG | metagpt.provider.base_llm:aask:126 - [{'role': 'user', 'parts': ['Please provide up to 2 necessary keywords related to your research topic for Google search. Your response must be in JSON format, for example: ["keyword1", "keyword2"].']}] 2024-03-19 13:08:38.442 | INFO | metagpt.utils.cost_manager:update_cost:57 - Total running cost: $0.000 | Max budget: $10.000 | Current cost: $0.000, p rompt_tokens: 37, completion_tokens: 9 2024-03-19 13:08:40.106 | DEBUG | metagpt.actions.research:run:138 - ### Requirements

  1. The keywords related to your research topic and the search results are shown in the "Search Result Information" section.
  2. Provide up to 4 queries related to your research topic base on the search results.
  3. Please respond in the following JSON format: ["query1", "query2", "query3", ...].

Search Result Information

Keyword: Computer Vision

Search Result: [{'title': 'Computer vision - Wikipedia', 'link': 'https://en.wikipedia.org/wiki/Computer_vision', 'snippet': 'Computer vision covers the core technology of automated image analysis which is used in many fields. Machine vision usually refers to a process of combining\xa0...'}, {'title': 'What is Computer Vision? | IBM', 'link': 'https://www.ibm.com/topics/computer-vision', 'snippet': 'Computer vision is a field of artificial intelligence (AI) enabling computers to derive information from images, videos and other inputs.'}, {'title': 'What Is Computer Vision? | Microsoft Azure', 'link': 'https://azure.microsoft.com/en-us/resources/cloud-computing-dictionary/what-is-computer-vision', 'snippet': 'Computer vision. Computer vision is a field of computer science that focuses on enabling computers to identify and understand objects and people in images and\xa0...'}, {'title': 'Computer Vision: What it is and why it matters | SAS', 'link': 'https://www.sas.com/en_us/insights/analytics/computer-vision.html', 'snippet': 'Computer vision can help agencies perform predictive maintenance by analyzing equipment and infrastructure images to make better decisions on which of these\xa0...'}, {'title': 'Azure AI Vision with OCR and AI | Microsoft Azure', 'link': 'https://azure.microsoft.com/en-us/products/ai-services/ai-vision', 'snippet': 'Accelerate computer vision development with Microsoft Azure. Unlock insights from image and video content using OCR, object detection, and image analysis.'}, {'title': 'Everything You Ever Wanted To Know About Computer Vision. | by ...', 'link': 'https://towardsdatascience.com/everything-you-ever-wanted-to-know-about-computer-vision-heres-a-look-why-it-s-so-awesome-e8a58dfb641e', 'snippet': "Computer vision also plays an important role in facial recognition applications, the technology that enables computers to match images of people's faces to\xa0..."}, {'title': 'Azure AI Vision documentation - Quickstarts, Tutorials, API Reference', 'link': 'https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/', 'snippet': 'The cloud-based Azure AI Vision API provides developers with access to advanced algorithms for processing images and returning information.'}, {'title': 'What is Computer Vision? - Image recognition AI/ML Explained - AWS', 'link': 'https://aws.amazon.com/what-is/computer-vision/', 'snippet': 'Computer vision applications use artificial intelligence and machine learning (AI/ML) to process this data accurately for object identification and facial\xa0...'}]

Keyword: Object Detection

Search Result: [{'title': 'Object detection - Wikipedia', 'link': 'https://en.wikipedia.org/wiki/Object_detection', 'snippet': 'Object detection ... Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic\xa0...'}, {'title': 'Object Detection | Papers With Code', 'link': 'https://paperswithcode.com/task/object-detection', 'snippet': 'Object Detection is a computer vision task in which the goal is to detect and locate objects of interest in an image or video. The task involves identifying\xa0...'}, {'title': 'What Is Object Detection? - MATLAB & Simulink', 'link': 'https://www.mathworks.com/discovery/object-detection.html', 'snippet': 'Object detection algorithms typically leverage machine learning or deep learning to produce meaningful results. When humans look at images or video, we can\xa0...'}, {'title': 'https://github.com/tensorflow/models/tree/master/r...', 'link': 'https://github.com/tensorflow/models/tree/master/research/object_detection'}, {'title': 'What is Object Detection? The Ultimate Guide.', 'link': 'https://blog.roboflow.com/object-detection/', 'snippet': 'Aug 22, 2023 ... Object detection can be used for any problem where [...snip...]

iorisa commented 3 months ago

It seems still running, since 'save report to ...' does not present in the log.

async def main():
    topic = "dataiku vs. datarobot"
    role = Researcher(language="en-us")
    await role.run(topic)
    print(f"save report to {RESEARCH_PATH / f'{topic}.md'}.")
vmsysadm commented 3 months ago

The log that's provided is truncated - in the command line I'm attempting to ask to research "meat vs icecream" question, but the output indicates that the research topic is ["Computer Vision", "Object Detection"] or something equally irrelevant.

(metagpt) PS C:\Users\user\code\MetaGPT> python -m metagpt.roles.researcher --help 2024-03-20 07:13:31.678 | INFO | metagpt.const:get_metagpt_package_root:29 - Package root set to C:\Users\user\code\MetaGPT INFO: Showing help with the command 'researcher.py -- --help'.

NAME researcher.py

SYNOPSIS researcher.py TOPIC

POSITIONAL ARGUMENTS TOPIC Type: str

FLAGS --language=LANGUAGE Default: 'en-us'

NOTES You can also use flags syntax for POSITIONAL ARGUMENTS

(metagpt) PS C:\Users\user\code\MetaGPT> python -m metagpt.roles.researcher "meat vs icecream" 2024-03-20 07:14:23.176 | INFO | metagpt.const:get_metagpt_package_root:29 - Package root set to C:\Users\user\code\MetaGPT 2024-03-20 07:14:28.451 | INFO | main:_act:56 - David(Researcher): to do CollectLinks(David) ["Machine Learning", "Natural Language Processing"] 2024-03-20 07:14:29.479 | INFO | metagpt.utils.cost_manager:update_cost:57 - Total running cost: $0.000 | Max budget: $10.000 | Current cost: $0.000, prompt_tokens: 37, completion_tokens: 10

[truncated]