Test batch norm, dropout, batch size, l2 weight

[ktrapeznikov]

(you can use a subsampled dataset to speed up experiment subsample by 8)

Ablation study, remove or add one element at a time and see how skill changes:

• dropout
• batch norm
• add weight decay (usually small 1e-6)

Effect of batch size on performance try scaling it down (but you may have to reduce number of epochs to get number of steps)

Effect of lr?

Generate performance plot for various parameter values