Parsed final answer in AgentFinish / TaskOutput often incomplete

[gontsharuk]

When determining the output of an agent action or a task, CrewAI relies on the relevant output to follow after the string "Final Answer:" in the complete LLM answer (implemented in crewai.agents.parser). Unfortunately, very often (say in 30% of the cases in our application) a lot of relevant information comes before "Final Answer:" (e.g. a complete markdown document) and these are then only briefly referenced in the final paragraph of the LLM answer after "Final Answer:", e.g.:

### Comprehensive User Research Report

#### 1. Introduction
**Product Name:** MyProduct
...
<many markdown lines left out>
...

Final Answer: The complete user research report, user personas, and wireframes/prototypes for MyProduct have been developed based on detailed user research methodologies and insights gathered from various techniques.

In this example the value of ["output"] in the return_values attribute of the corresponding AgentFinish object will be the text after "Final Answer:":

The complete user research report, user personas, and wireframes/prototypes for MyProduct have been developed based on detailed user research methodologies and insights gathered from various techniques.

Instead, it should contain the complete Markdown document (Comprehensive User Research Report in above example).

It looks like this behavior is largely due to the way the LLM prompts are constructed from the snippets in crewai/translations/en.json. I modified those locally and was able to bring down the rate of unsatisfactory LLM answers down to like 5-10%. But I feel that those prompt snippets can be improved even further to ensure the correct structure in nearly 100% of the cases.

[aliogh]

Any updates on this? I have the same problem

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.