bquast / rnn

Recurrent Neural Networks in R
https://qua.st/rnn
73 stars 28 forks source link

lstm and gru, not learning #11

Closed DimitriF closed 8 years ago

DimitriF commented 8 years ago

I cleaned the doc every where, there should be still some stuff to change though.

The seq_to_seq option is implemented everywhere, basically, the user can supply a X and Y with different time dim, including dim(Y)[2] == 1 (many to one). The two object are modified at the beginning of the training (or at the beginning of the prediction if predictr) and only the output starting at dim(X)[2] has effect, for backpropagation or as output for the prediction. also the input after dim(X)[2] is set to zero so there is no input once the input sequence is finish.

the lstm and gru are doing stuf but not learning,

I change the update of the weight into 2 part, sgd for stochastic gradient descent and possibility to choose other techniques like adagrad or rmsprop, I did one test but it didn't work, still it's implemented so it's like for lstm

I put a lot of files in the vignette folder, example but not in Rmd format, just R files

Also I added a shiny app with now 4 toy datasets: binary addition, cosinus, finance data and oscilation. The finance app didn't work anymore because time_dim was extracted during the training so I change that as well