search

Lightning-AI / torchmetrics

Torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications.
https://lightning.ai/docs/torchmetrics/
Apache License 2.0
2.08k stars 398 forks source link

Add `ChrF++` #634

Closed stancld closed 5 days ago

stancld commented 2 years ago

🚀 Feature

Add ChrF++, a text metric used for NMT evaluation.

Sources: Paper - CHRF++: words helping character n-grams Repo / metric is also implemented in sacrebleu

Motivation

Character n-gram F-score (CHRF) is shown to correlate very well with human relative rankings of different machine translation outputs, especially for morphologically rich target languages. (Popović, 2017)

Additional context

I've already started working on this metric, I'll open a PR soon. :]

Borda commented 1 month ago

I have contacted the author of m-popovic/chrF@master/chrF++.py so we may ask her if she is willing to change the license (so we could keep implementation dropped in #2668) or contribute directly to TM or let's import https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py

SkafteNicki commented 5 days ago

Re closing due to PR https://github.com/Lightning-AI/torchmetrics/pull/2701