I created the model object from the start (almost...). then this object in this list are called and update all along the function.
I also create the logic for user define function at different point:
in the sample loop to apply the learning rate, I called it "loss" but I am not sure of the vocabulary
in the end of the epoch loop, two function, one with side effect on the model (for the annealing for example), one without (for print output but I want to use it also for animation). In both case, it's possible to supply several function in a given order for whatever purpose.
the "..." R argument is automatically append inside the model object so it should be possible to add user define argument, a test set for example to print test error at each epoch.
This is the first step before integrating of the LSTM, there is then several steps where we can call other functions specific for both:
initialisation (of the weight and bias matrices)
feed forward (calling the already existing predictr/predict_rnn and the needed predict_lstm)
back propagation (same idea than feed forward)
update (same idea than initialisation)
there will be a trick between feed forward and back propagation in order to access the error but I think the logic is the same for rnn and lstm so it should do.
Ok the travis check fails because it is using the CRAN version of sigmoid which doesn't have the tanh_output_to_derivive function yet. I will submit the latest version of sigmoid to CRAN package now.
lots of change here:
I created the model object from the start (almost...). then this object in this list are called and update all along the function.
I also create the logic for user define function at different point:
This is the first step before integrating of the LSTM, there is then several steps where we can call other functions specific for both:
there will be a trick between feed forward and back propagation in order to access the error but I think the logic is the same for rnn and lstm so it should do.