Store computed metrics in a dataframe

confident-ai / deepeval

The LLM Evaluation Framework

https://docs.confident-ai.com/

Apache License 2.0

2.39k stars 166 forks source link

Store computed metrics in a dataframe #595

Open piseabhijeet opened 4 months ago

piseabhijeet commented 4 months ago

Objective: I have many test cases - query, response, context trios in a pandas dataframe. I create an llm test case per trio.. however the outcome of bulk testing locally is a json file.

Outcome: save the responses in the same dataframe while computing the scores. this should be asynchronously done. and the json file should be used only for validation purpose.

penguine-ip commented 3 months ago

Hey @piseabhijeet , can you show an example? I get what you need, but i'm not sure how would it write to the same dataframe.