facebookresearch / SentEval

A python tool for evaluating the quality of sentence embeddings.
Other
2.09k stars 309 forks source link

Hyperparameter tuning - discrepancy between readme and code? #73

Open maruker opened 5 years ago

maruker commented 5 years ago

The readme states

For probing tasks, we used an MLP with a Sigmoid nonlinearity and and tuned the nhid (in [50, 100, 200]) and dropout (in [0.0, 0.1, 0.2]) on the dev set.

However, in the code it looks like the parameters given by the user are always used. No tuning takes place and no predefined hyperparameters are loaded. Maybe I missed something?

Should I do hyperparameter tuning to get results that are comparable to the literature?