[Feat_Add] Addition of new LLM evals metric

aiplanethub / beyondllm

Build, evaluate and observe LLM apps

https://beyondllm.aiplanet.com/

Apache License 2.0

263 stars 40 forks source link

[Feat_Add] Addition of new LLM evals metric #32

Open tarun-aiplanet opened 7 months ago

tarun-aiplanet commented 7 months ago

Beyond LLM supports, 4 evaluation metrics: Context relevancy, Answer relevancy, Groundedness, and Ground truth.

We would be looking forward to add new evaluation metric support to evaluate LLM/RAG response

[ ] Faithfulness
[ ] Correctness

Or any other research based metric

adityasingh-0803 commented 4 weeks ago

I can work on lexical diversity

adityasingh-0803 commented 3 weeks ago

@tarun-aiplanet please assign it

tarun-aiplanet commented 3 weeks ago

lexical diversity

I have never heard such metrics to evaluate LLM. Can you provide the reference of research paper. Also you were assigned for Perplexity LLM. Is that done?