Investigate other regularization methods for deep models

[accosmin]

• insensitivity to parameter changes: penalize with the gradient magnitude of the layer's output to its paramters

[accosmin]

• L2-regularization on weights, but not on biases
• randomly multiply weights with N(1, I) - see deep learning book