search

Lightning-AI / torchmetrics

Machine learning metrics for distributed, scalable PyTorch applications.
https://lightning.ai/docs/torchmetrics/
Apache License 2.0
2.13k stars 408 forks source link

[Metrics] ROUGE #51

Closed justusschock closed 3 years ago

justusschock commented 4 years ago

🚀 Feature

Implement ROUGE

carmocca commented 3 years ago

@SeanNaren you could take this at some point, as you already have a working implementation relying on https://pypi.org/project/rouge-score/

carmocca commented 3 years ago

https://github.com/PyTorchLightning/lightning-transformers/blob/1c5c91f5b4962f9162bcbd41fee7a7ac5eae00a9/lightning_transformers/task/nlp/summarization/metric.py

SkafteNicki commented 3 years ago

Copy implementation from here: https://github.com/PyTorchLightning/lightning-flash/blob/master/flash/text/seq2seq/summarization/metric.py

randolph-john commented 3 years ago

A word of caution, the PyTorchLightning implementation of Rouge does not give the same results as the original PERL script. See here: https://github.com/google-research/google-research/issues/239. We also encountered this issue internally at Facebook.

Borda commented 3 years ago

@ananyahjha93 any update here? :rabbit:

SkafteNicki commented 3 years ago

@Borda if I am not wrong, @karthikrangasai would like to work on this issue when his Bleu PR has landed

karthikrangasai commented 3 years ago

@SkafteNicki yeah sure. I would like to work on this issue.