search

huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
https://huggingface.co/docs/evaluate
Apache License 2.0
2.04k stars 259 forks source link

Rouge-SU-4 (skipgram) #384

Open ahmed-moubtahij opened 1 year ago

ahmed-moubtahij commented 1 year ago

I was trying to migrate from https://github.com/tagucci/pythonrouge whose ROUGE-SU-4 has the highest variance on my summary evaluations (and so is an important metric in the PCA/eigenvector sense) but you don't provide it. Are there plans for it? Is there a reason why it wasn't or won't be implemented?

lvwerra commented 1 year ago

The ROUGE implementation in evaluate relies on Google's ROUGE repository. Feel free to open a PR if you'd like to add this as a feature.