search

yandex / faster-rnnlm

Faster Recurrent Neural Network Language Modeling Toolkit with Noise Contrastive Estimation and Hierarchical Softmax
Other
561 stars 138 forks source link

Could you supply optimal configurations for 1B benchmark? #32

Closed alexandres closed 8 years ago

alexandres commented 8 years ago

I'm trying to use faster-rnnlm on a 3B word dataset and would like to use the optimal hyperparameters you obtained in the One Billion Word benchmark.

In particular, the end of this section https://github.com/yandex/faster-rnnlm#one-billion-word-benchmark contains the sentence:

Note. We took the best performing models from the previous and added maxent layer of size 1000 and order 3.

Is there any way you can provide what those hyperparameters are for each of the three models graphed?

Thanks!

akhti commented 8 years ago

Hi! Check out results file: https://github.com/yandex/faster-rnnlm/blob/master/doc/RESULTS.md It contains command line arguments for the benchmark.

alexandres commented 8 years ago

Thank you. This line in the README should be corrected:

Note. We took the best performing models from the previous and added maxent layer of size 1000 and order 3.

In the RESULTS.md file, the best performing models use --direct-order 4 --direct 1000, not order 3.