Difference in parameters between config.yaml and paper

[Merlin-721]

Config.yaml vs Paper

• lambda_kl: 1 vs 0.01
• batch_size: 128 vs 256
• dropout_rate: 0 vs 0.5 on all layers

Anyone know the reason for this? Are these updated values that should be used?

Most notably the batch_size would affect the number of training iterations, where the current 200k would only perform half the training with 128 samples per batch, vs 256 samples per batch.

[jjery2243542]

Hi, I found that some hyperparameters (ex. dropout) are unnecessary so I updated the number in this repo. Feel free to tune the number.