Add a new `BERTScore` descriptor to Evidently

evidentlyai / evidently

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

https://www.evidentlyai.com/evidently-oss

Apache License 2.0

5.46k stars 603 forks source link

Add a new `BERTScore` descriptor to Evidently #1317

Closed elenasamuylova closed 4 weeks ago

elenasamuylova commented 2 months ago

About Hacktoberfest contributions: https://github.com/evidentlyai/evidently/wiki/Hacktoberfest-2024

Description:

Implement BERTScore - a metric that computes a similarity score for each token in the candidate sentence with each token in the reference sentence.

Paper: https://arxiv.org/abs/1904.09675

Check existing SemanticSimilarity descriptor as a reference implementation.

csking101 commented 1 month ago

Hi, could I take this up?

elenasamuylova commented 1 month ago

Hi @csking101, there is already a contribution pending to be merged for this one: https://github.com/evidentlyai/evidently/pull/1335