a1da4 commented 5 years ago

0. Paper

@inproceedings{yu-etal-2015-casict, title = "{CASICT}-{DCU} Participation in {WMT}2015 Metrics Task", author = "Yu, Hui and Ma, Qingsong and Wu, Xiaofeng and Liu, Qun", booktitle = "Proceedings of the Tenth Workshop on Statistical Machine Translation", month = sep, year = "2015", address = "Lisbon, Portugal", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/W15-3053", doi = "10.18653/v1/W15-3053", pages = "417--421", }

1. What is it?

They proposed a novel evaluation metric, DPMFcomb. Achieved SotA in WMT15 metric task.

2. What is amazing compared to previous studies?

Recent work used only syntax information or lexical information. They proposed DPMF which uses both information.

syntax similarity: Dependency Paring Model
lexical similarity: unigram Fscore

In this paper, they combined metric which is named as DPMFcomb.

DPMF
REDp
ENTFp
Asiya

3. Where is the key to technologies and techniques?

Combine DPMF and...

REDp: calculate F-score using dependency tree of reference and hypothesis
ENTFp: calculate unigram F-score to reflect the fluency of translations
Asiya: default has 55 metrics set, which includes lexical, syntactic and semantic similarity

Model

They used SVM-rank

4. How did validate it?

They tried WMT14 metric task.

Baseline

BLEU
TER
METEOR

SotA in WMT14

system level: VRTA-W
sentence level: BEER
BOTH: DISCOTK-PARTY-TUNED

5. Is there a discussion?

6. Which paper should read next?