Future-House / paper-qa

High accuracy RAG for answering questions from scientific documents with citations
Apache License 2.0
5.45k stars 517 forks source link

o1 Agent Error on Complex Types #388

Open whitead opened 4 days ago

whitead commented 4 days ago

We're failing on o1 agents because of a complex types error:

 NotImplementedError: Complex types are not yet supported. Failed on: FunctionInfo(name='paper_search', description='Search for papers to increase the paper count.\n\nRepeat previous calls with the same
           query and years to continue a search. Only repeat a maximum of twice.\nThis tool can be called concurrently.\nThis tool introduces novel papers, so invoke this tool when just beginning or when         
           unsatisfied with the current evidence.', parameters=Parameters(type='object', properties={'query': {'description': "A search query, which can be a specific phrase, complete sentence, or general        
           keywords, e.g. 'machine learning for immunology'. Also can be given search operators.", 'title': 'Query', 'type': 'string'}, 'min_year': {'anyOf': [{'type': 'integer'}, {'type': 'null'}],              
           'description': 'Filter for minimum publication year, or None for no minimum year. The current year is 2024.', 'title': 'Min Year'}, 'max_year': {'anyOf': [{'type': 'integer'}, {'type': 'null'}],       
           'description': 'Filter for maximum publication year, or None for no maximum year. The current year is 2024.', 'title': 'Max Year'}}, required=['query', 'min_year', 'max_year']))     

We could either (1) just drop year for now or (2) give our React parsing some love and come up with a solution for this

jamesbraza commented 4 days ago

Oh the solution here is to use JSON or XML parsing. Our STR template has this vulnerability. You would place {"tool_description_method": ToolDescriptionMethods.JSON} in the agent_config

whitead commented 4 days ago

Got it - we should get this as default somehow so we use o1 on CLI:

pqa --agent.agent_llm=o1-mini --agent.agent_type=ldp.agent.ReActAgent ask "How are quantum computing and MD related?"