Marker-Inc-Korea / RAGchain

Extension of Langchain for RAG. Easy benchmarking, multiple retrievals, reranker, time-aware RAG, and so on...
Apache License 2.0
277 stars 28 forks source link

implement EM answer metric #437

Closed Eastsidegunn closed 9 months ago

Eastsidegunn commented 9 months ago

Now we have EM score (answer evaluation with ground truths) change answer metric solution input for multiple ground truth values.

our function will calculate max of EM scores between each gt values and answer(1 or 0)

EM use normalizer method of BaseAnswerMetric.(from SQuAD)

close #435