They propose a novel automatic evaluation metric based on the dependency parsing model.
2. What is amazing compared to previous studies?
They do not rely on sub-structures defined by human.
Previous syntax-based works obtain the similarity by comparing the sub-structures from trees of reference and hypothesis defined by human.
3. Where is the key to technologies and techniques?
Dependency Parsing Model
DPM was trained using the reference tree and shift-reduce algorithm.
After that, it calculates DPM score via predicting hypothesis trees.
F score
The DPM method is syntax-based metric.
They used F-score to consider the lexicon-based metric.
DPMF score
Finally, DPMF score was computed.
4. How did validate it?
They tried WMT12-14 metric task (system-level and sentence-level).
In system-level, their model achieved SotA result.
a1da4
commented
4 years ago
0. Paper
An Automatic Machine Translation Evaluation Metric Based on Dependency Parsing Model
1. What is it?
They propose a novel automatic evaluation metric based on the dependency parsing model.
2. What is amazing compared to previous studies?
They do not rely on sub-structures defined by human. Previous syntax-based works obtain the similarity by comparing the sub-structures from trees of reference and hypothesis defined by human.
3. Where is the key to technologies and techniques?
Dependency Parsing Model
DPM was trained using the reference tree and shift-reduce algorithm.
After that, it calculates DPM score via predicting hypothesis trees.
F score
The DPM method is syntax-based metric. They used F-score to consider the lexicon-based metric.
DPMF score
Finally, DPMF score was computed.
4. How did validate it?
They tried WMT12-14 metric task (system-level and sentence-level). In system-level, their model achieved SotA result.
5. Is there a discussion?
6. Which paper should read next?