Persistent error when attempting to use tool with the correct input format

P7201 commented 2 months ago

The final agent and tool in my crew is having an issue where it will not accept the input from its preceding crew member. The input is coming in as a JSON object and seems to be formatted properly but the tool will not accept it. Here is the Output:

Action Input:

{
  "coworker": "Expert Query Analyzer and Classifier",
  "question": "I am encountering a persistent error when attempting to use the Document_Synthesis tool with the correct input format. The error states that the Action 
Input is not a valid key, value dictionary, despite the format appearing correct. Can you help me understand why this error is occurring and how to resolve it?",      
  "context": "I am trying to synthesize information from a document titled 'The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf' using the Document_Synthesis tool. The input provided is a JSON dictionary with the query and the document details, which is the expected format for the tool. However, the tool is 
returning an error message stating that the input is not valid. Here is the input I have been using: {\"query\": \"What are some common variables used in studies regarding human error-based aviation accidents\", \"documents\": [{\"title\": \"The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf\", \"path\": \"C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf\"}]}"
}

Final Answer:

I apologize for the inconvenience, but due to a persistent error with the tools provided, I am unable to access the document and synthesize the information required to answer the query regarding common variables used in studies of human error-based aviation accidents. My goal was to provide a detailed synthesis report that addressed the query comprehensively, but without the ability to analyze the document, I cannot fulfill this task. If you have any further questions or need assistance with a different query, please let me know, and I will do my best to assist you.

joaomdmoura commented 2 months ago

Oh that is an interesting one, can you share more of the logs so we can debug a little further?

P7201 commented 2 months ago

Yes, I have made a few changes to the tools, and tasks assigned to the various agents. I can share the tools and tasks files and the full output of the thought process.

P7201 commented 2 months ago

The problem seems to be the agent is getting data passed into its tool with json formatting for example: Action Input:

{
  "query": "What are some common variables used in studies regarding human error-based aviation accidents",
  "documents": [
    {
      "title": "The Role of Human Factors in Aviation Ground Operation-Related Accidents",
      "path": "C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf"
    }
  ]
}

Thought: It seems there was an error in the action input format. I need to correct the input format to ensure it is a valid key-value dictionary as expected by the Document_Synthesis tool.

How do I prevent this formatting and just pass the inputs as a key value pair?

joaomdmoura commented 2 months ago

Weird we should threat for that, can you share a little more of the log surrounding that error?

P7201 commented 2 months ago

The first agent loops through a directory of documents and summarizes them. The second agent takes a user query and compares it to the document summaries to determine which documents are needed to answer the question. The names and paths of the necessary documents are passed to a third agent to process for analysis. I'm working on building out a systematic literature review and meta analysis tool. I'd like to go over with you.

Finished chain. [DEBUG]: == [Expert Query Analyzer and Classifier] Task output: ```json { "need_single_doc": true, "need_multiple_docs": false, "documents": [.

Finished chain. [DEBUG]: == [Expert Query Analyzer and Classifier] Task output: ```json { "need_single_doc": true, "need_multiple_docs": false, "documents": [ { "title": "The Role of Human Factors in Aviation Ground Operation-Related Accidents", "path": "C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf" } ] }

[DEBUG]: == Working Agent: Expert Integrative Synthesizer [INFO]: == Starting Task: Wait until the query_analysis_agent has completed their task. Take the list of documents from the query_analysis_agent to perform your task to synthesize the contents of multiple documents in order to provide a detailed and comprehensive answer to the query: 'What are some common variables used in studies regarding human error-based aviation accidents '. This task involves:

Receiving the necessary document paths and titles from the query_analysis_agent to answer the query
Analyzing each document to extract key information, themes, and data points that are directly relevant to the query.
Comparing and contrasting the findings across different documents to identify commonalities, discrepancies, and unique insights.
Integrating these insights into a coherent narrative that addresses the query's requirements, highlighting how each piece of information contributes to understanding the broader topic.
Utilizing advanced NLP techniques to ensure that the synthesis is not only comprehensive but also presents the information in an easily digestible format for the end-user.

Entering new CrewAgentExecutor chain... Thought: Before I can synthesize the information from the provided document, I need to extract key information, themes, and data points that are directly relevant to the query about common variables used in studies regarding human error-based aviation accidents. Since I have been provided with the document's title and path, I can proceed to analyze the document using the Document_Synthesis tool.

Action: Document_Synthesis

Action Input:

{
  "query": "What are some common variables used in studies regarding human error-based aviation accidents",
  "documents": [
    {
      "title": "The Role of Human Factors in Aviation Ground Operation-Related Accidents",
      "path": "C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf"
    }
  ]
}

avri-schneider commented 2 months ago

@P7201, @joaomdmoura The problem is the Action Input is not passed verbatim, rather wrapped inside a ```json...``` block. As a workaround, you may want to specifically ask in the task not to do so - i.e., add the following to the task:

IMPORTANT INSTRUCTIONS:
------------------------------
- Action Input, although a JSON dictionary, must *never* be surrounded by backticks!
  It must be passed verbatim as a JSON dictionary.
- "```json ... ```" is not allowed!!!. This is critical, as the tool will not be able to function correctly if this is done.

I find that the agent doesn't always adhere to these instructions on the first attempt, but corrects itself on the second, once tool execution fails. IMO it is better to handle it in the output parser.

Bitsy-Chuck commented 2 weeks ago

@avri-schneider for me it is like

It appears there is a persistent issue with the format of the input. I will ensure that the format strictly adheres to the expected key-value structure. Let's break down the task and ensure each key-value pair is correctly formatted within the input dictionary.

Action: Delegate work to coworker
Action Input:
{
    "task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.",
    "context": {
        "stock_name": "SPAL",
        "exchange": "NSE",
        "end_date": "2023-12-31",
        "investment_horizon": "long",
   }
}

Still getting Error: the Action Input is not a valid key, value dictionary.

avri-schneider commented 2 weeks ago

Can you try the following Action Input?

{ "task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.", "context": "{\"stock_name\": \"SPAL\", \"exchange\": \"NSE\", \"end_date\": \"2023-12-31\", \"investment_horizon\": \"long\"}" }

On Sun, Jun 23, 2024 at 11:06 PM Ojasv Singh @.***> wrote:

@avri-schneider https://github.com/avri-schneider for me it is like

It appears there is a persistent issue with the format of the input. I will ensure that the format strictly adheres to the expected key-value structure. Let's break down the task and ensure each key-value pair is correctly formatted within the input dictionary.

Action: Delegate work to coworker Action Input: { "task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.", "context": { "stock_name": "SPAL", "exchange": "NSE", "end_date": "2023-12-31", "investment_horizon": "long", } }

Still getting Error: the Action Input is not a valid key, value dictionary.

— Reply to this email directly, view it on GitHub https://github.com/joaomdmoura/crewAI/issues/547#issuecomment-2185309770, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTYRHMS7BXAUJP6PANEO33ZI4TFNAVCNFSM6AAAAABHCIJJ5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGMYDSNZXGA . You are receiving this because you were mentioned.Message ID: @.***>

joaomdmoura / crewAI

Persistent error when attempting to use tool with the correct input format #547

Can you try the following Action Input?

{ "task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.", "context": "{\"stock_name\": \"SPAL\", \"exchange\": \"NSE\", \"end_date\": \"2023-12-31\", \"investment_horizon\": \"long\"}" }