search

duolingo / halflife-regression

MIT License
494 stars 88 forks source link

Evaluation results #3

Closed ManSchri closed 7 years ago

ManSchri commented 7 years ago

I have noticed the evaluation results for the half-life regression model are slightly different each time I run the code. So I was wondering how the evaluation results presented in the paper are determined.

Are the results given in table 2 of the paper based on one run or are they based on multiple runs (using the best result or the average or something else)?

Thanks!

burrsettles commented 7 years ago

It is a property of stochastic gradient descent that you will get slightly different answers for each run. However, on this particular data set, results didn't vary more than 1%, and were pretty stable, so we reported only a single pass.

If you're more comfortable doing multiple passes, you can easily modify the code to do that, and average over them...