Print the 'thought process' transcript for Agent

deepset-ai / haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

https://haystack.deepset.ai

Apache License 2.0

16.67k stars 1.83k forks source link

Print the 'thought process' transcript for Agent #4286

Closed TuanaCelik closed 1 year ago

TuanaCelik commented 1 year ago

@vblagoje and @julian-risch - I wanted to create this feature request after using the agent for a bit. As a user I've noticed that although restult["transcript"] is useful, the most interesting and useful part for me to observe is what you can reach in the second part of the transctipt. Which I can (as far as I can tell) only get as follows: result["transcript"].split('---')[1]. It works, but I think it would be nice to think of a name for this part of the transcript, and let users access it directly

tholor commented 1 year ago

In addition, I'd find it helpful to:

see the transcript being printed on-the-fly once the agent enters a new step (rather than just at the very end). This can help users a lot to understand the actual steps an agent takes + facilitates debugging if the agents crashes before returning the final transcript
have a clearer visual separation between the prompt and the steps of the "reasoning chain" afterwards

vblagoje commented 1 year ago

We had this on our to-do list, but I agree it should be prioritized. Now that we are paying more attention to user experience, why don't we also implement streaming while at it? It's not super-hard and would further improve an impression of the agent's progress. Of course, it would work only for OpenAI models, but I think, it is possible to even for HF models now (we can do them later).

TuanaCelik commented 1 year ago

@tholor and @vblagoje - If we're aiming for Agents in 1.15, at least as a first step having this result["transcript"].split('---')[1] accessible as a value like result["observation"] or similar (I leave the naming to you) would make the tutorial explanation clean. (or result["transcript"]["observation"]?) -- just as a sidenote for something that could be implemented quicker..

julian-risch commented 1 year ago

This is issue can be closed. It's implemented and shown in the tutorial: https://haystack.deepset.ai/tutorials/23_answering_multihop_questions_with_agents