search

bloomsburyai / question-generation

Neural text-to-text question generation
MIT License
217 stars 52 forks source link

Fine-tuning the hyperparameters #23

Closed ayush2051 closed 5 years ago

ayush2051 commented 5 years ago

Hi Tom, Did you fine-tune any of the hyper-parameters that u have shared in the FLAGS file as I tried with the same and got a BLEU score of 14.17, NLL- 40.33.; with the GLOVE vocab, PG set to FALSE. Can u let me know the exact split of the dev and test set samples that you used for the evaluation as this high score may be biased due to some different split I took?

tomhosking commented 5 years ago

Hi, I'd strongly recommend using the split from here as it has become the standard split in the literature: https://github.com/tomhosking/squad-du-split

I didn't do a full hyperparameter optimisation, but did play around with them a bit during development.

ayush2051 commented 5 years ago

Thanks a lot Tom!! will try a few combinations on the top of that.