The current implementation of weight/dropout regularizer seems to be not proper for cross-entroy loss.
If I understand it correctly, the factor 2 in length scale is removed in this code (you set \lambda = (l*2)(1-p) / (\tau N)).
This is proper for MSE loss. But the form of cross entroy loss is exactly the same as the negative log-likelihood version of Euclidean loss. So I think the factor 2 can not be removed in this case.
The current implementation of weight/dropout regularizer seems to be not proper for cross-entroy loss. If I understand it correctly, the factor 2 in length scale is removed in this code (you set \lambda = (l*2)(1-p) / (\tau N)).
This is proper for MSE loss. But the form of cross entroy loss is exactly the same as the negative log-likelihood version of Euclidean loss. So I think the factor 2 can not be removed in this case.
Thanks.