Reading: CASICT-DCU Participation in WMT2015 Metrics Task

[a1da4]

0. Paper

@inproceedings{yu-etal-2015-casict, title = "{CASICT}-{DCU} Participation in {WMT}2015 Metrics Task", author = "Yu, Hui and Ma, Qingsong and Wu, Xiaofeng and Liu, Qun", booktitle = "Proceedings of the Tenth Workshop on Statistical Machine Translation", month = sep, year = "2015", address = "Lisbon, Portugal", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/W15-3053", doi = "10.18653/v1/W15-3053", pages = "417--421", }

1. What is it?

They proposed a novel evaluation metric, DPMFcomb. Achieved SotA in WMT15 metric task.

2. What is amazing compared to previous studies?

Recent work used only syntax information or lexical information. They proposed DPMF which uses both information.

• syntax similarity: Dependency Paring Model
• lexical similarity: unigram Fscore

In this paper, they combined metric which is named as DPMFcomb.

• DPMF
• REDp
• ENTFp
• Asiya

3. Where is the key to technologies and techniques?

Combine DPMF and...

• REDp: calculate F-score using dependency tree of reference and hypothesis
• ENTFp: calculate unigram F-score to reflect the fluency of translations
• Asiya: default has 55 metrics set, which includes lexical, syntactic and semantic similarity

Model

They used SVM-rank

4. How did validate it?

They tried WMT14 metric task.

Baseline

• BLEU
• TER
• METEOR

SotA in WMT14

• system level: VRTA-W
• sentence level: BEER
• BOTH: DISCOTK-PARTY-TUNED

5. Is there a discussion?

6. Which paper should read next?

• DPMF
• RED
• ENT
• Asiya

[a1da4]

43 Blend

[a1da4]

47 DPMF