GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

Support for iBLEU for paraphrasing #85

Open ratishsp opened 2 years ago

ratishsp commented 2 years ago

iBLEU [1] is an important metric for evaluation of paraphrase generation (for ex [2]). iBLEU measures similarity with reference while penalizing similarity with the input sentence. It will be great if this metric is supported.

  1. Sun, H., & Zhou, M. (2012, July). Joint learning of a dual SMT system for paraphrase generation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 38-42).
  2. Hosking, T., & Lapata, M. (2021, August). Factorising Meaning and Form for Intent-Preserving Paraphrasing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 1405-1418).