evaluation and dataset analysis

qe-team / marmot

MARMOT - the open source framework for feature extraction and machine learning, designed to estimate the quality of Machine Translation output

ISC License

21 stars 7 forks source link

Open chrishokamp opened 9 years ago

chrishokamp commented 9 years ago

we need utilities for comparing datasets and analyzing the statistical significance of results.

Questions that we should be able to answer

which tokens in the test set are OOV with respect to the training data?
what is the statistical significance of our performance on a dev set -- i.e. are we doing better than random, or the most frequent class baseline?