search

ghpaetzold / questplusplus

Pipelined quality estimation.
49 stars 14 forks source link

HTER prediction systems. #32

Closed znavoyan closed 6 years ago

znavoyan commented 8 years ago

Hello, I was reviewing Pearson correlation results of HTER prediction on WMT-13 Task1.1, which I attached to the post. It shows system name, correlation coefficient and MAE. Can you please suggest where can I find information on the mentioned systems (what features, machine learning algorithms they used)? Is it known information at least for TOP 5 systems?

System r MAE DCU-SYMC-rc 0.595 0.135 SHEFMIN-FS 0.575 0.124 DCU-SYMC-ra 0.572 0.135 CNGL-SVRPLS 0.560 0.133 CMU-ISL-noB 0.516 0.138 CNGL-SVR 0.508 0.138 CMU-ISL-full 0.494 0.152 fbk-uedin-extra 0.483 0.144 LIMSI-ELASTIC 0.475 0.133 SHEFMIN-FS-AL 0.474 0.130 LORIA-INCTRA-CONT 0.474 0.148 fbk-uedin-rsvr 0.464 0.145 LORIA-INCTRA 0.461 0.148 baseline 0.451 0.148 TCD-CNGL-OPEN 0.329 0.148 TCD-CNGL-RESTR 0.291 0.152 UMAC-EBLEU 0.113 0.170

Thank you, Zaven.

carolscarton commented 8 years ago

Hi,

A summary of this information can be found on the "Findings of the 2013 Workshop on Statistical Machine Translation" paper (http://statmt.org/wmt13/pdf/WMT01.pdf).

Detailed information about the systems are usually provided by the authors in the shared task papers: http://statmt.org/wmt13/papers.html

For example, for the first systems, the correspondent papers are: DCU-SYMC: http://statmt.org/wmt13/pdf/WMT49.pdf SHEFMIN-FS: http://statmt.org/wmt13/pdf/WMT41.pdf

Kind Regards, Carol

znavoyan commented 8 years ago

Thank you!