bjerva / cwi18

Repository for https://www.aclweb.org/anthology/W18-0518/
Apache License 2.0
0 stars 0 forks source link

Compare to non-neural baseline #14

Closed jbingel closed 6 years ago

jbingel commented 6 years ago

Good news: with our simple features (word length, frequency, character perplexity, target/sentence similarity), random forest gets F1=0.69 on the German data, compared to roughly 0.75 with the best neural MTL model

jbingel commented 6 years ago

For English and Spanish, however, random forest seems to strongly outperform neural models:

neural RF
EN 0.70 0.78
ES 0.64 0.69
DE 0.75 0.69
jbingel commented 6 years ago

new features improved neural models:

neural RF
EN 0.74 0.79
ES 0.69 0.69
DE 0.76 0.69
bjerva commented 6 years ago

Nice!!!

jbingel commented 6 years ago

Yeah. For the final submission (especially for EN and ES) we could try an ensemble between neural and RF (and others...). I'll look to get this implemented.