Added parallel tool call evaluation to ToolCallAccuracy metric

explodinggradients / ragas

Supercharge Your LLM Application Evaluations 🚀

https://docs.ragas.io

Apache License 2.0

7.3k stars 746 forks source link

Added parallel tool call evaluation to ToolCallAccuracy metric #1687

Open sahusiddharth opened 4 days ago

sahusiddharth commented 4 days ago

Fixes #1658
Introduced the mode parameter in the ToolCallAccuracy class to allow selection between sequential and parallel evaluation modes.

shahules786 commented 3 days ago

I think this is a better solution for now, compared to introducing new metric just for parallel like https://github.com/explodinggradients/ragas/pull/1665. But ideally what we need is an interface where use could both in single run. Ie there can be tool calls in single agentic run that should be made in parelle and ones that could be made in sequence. I think we will need another iteration on agentic evals to get there @jjmachan should we consider merging this for now?