[feature/improvements] List of metrics to implement for NLP

NASA-IMPACT / evalem

An evaluation framework for your large model pipelines

0 stars 0 forks source link

[feature/improvements] List of metrics to implement for NLP #7

Open NISH1001 opened 1 year ago

NISH1001 commented 1 year ago

Following are the metrics we need to implement based on jury

[x] Bleu score
[ ] sequence evaluation (NER)
[x] ROUGE
[x] Meteor
[ ] Perplexity

All these metrics could be derived through evalem.metrics._base.JuryBasedMetric (see evalem.metrics.basics module for some general implementation)

cc: @muthukumaranR @xhagrg @code-geek