search

explodinggradients / ragas

Supercharge Your LLM Application Evaluations 🚀
https://docs.ragas.io
Apache License 2.0
7.3k stars 746 forks source link

Update tool call accuracy. #1665

Open MattZ-99 opened 1 week ago

MattZ-99 commented 1 week ago

ref: #1658.

Two updates:

shahules786 commented 1 week ago

As I just commented here I am wondering if having another metric for parallel tool call is the best option or if there is a good interface to do it inside toolcallaccuracy itself. @jjmachan @MattZ-99 Thanks a lot for the PR ❤️

MattZ-99 commented 1 week ago

Yeah. Really good point, as current ToolCallParallelAccuracy is highly inherited from the original ToolCallAccuracy.

shahules786 commented 1 week ago

Hey @MattZ-99 An update on our side, as you agreed we want to add both in one interface. ie to to toolcallaccuracy itself. But we need to find the best way for user's specify which calls should be parallel and in sequence. We will work together and make it happen, if you have any ideas please feel to share it here.

MattZ-99 commented 4 days ago

Hi @shahules786 , I have thought about it. A straightforward idea is add a parameter like ordered_tool_call which controls 'parallel' or 'sequencial' function call. What do u think about it? Or could you share your code?