Open jonny-d opened 6 years ago
Without any regularization I personally found that uniform sampling gave faster convergence, but was more unstable and blew up (see my issue #39). I also tried xavier initialization and that seemed to be more stable.
Hello, sorry for the delayed response.
I have achieved pretty good performance using a normal distribution for the initial weights. Here is a link to my Tensorflow Implementation
@jonnykira I found like you that they used weight norm in the paper which I initially glossed over/isn't in the code base. This turned out to be what I needed to use.
@jonnykira hello, I trained the model on my dataset, this generates me three files : model.data, model.index, model.meta
How can i generates the 15 .npy weight files (0.npy, 1.npy, ..., 14.npy) to test the sentiment analysis code (as in in https://github.com/openai/generating-reviews-discovering-sentiment) ?
thank you !
Hello,
Thank you very much for sharing this code. I am attempting to re-train a model like this from scratch and was wondering which weight initialization method was used for training the model?
Thanks, Jonny