search

yabata / pyrenn

A Recurrent Neural Network Toolbox for Python and Matlab
GNU General Public License v3.0
99 stars 42 forks source link

Update Train_LM with minimum error step criterium #8

Closed mc10011 closed 5 years ago

mc10011 commented 5 years ago

The proposed changes help in two ways:

  1. Prevent training from running forever when error improves slightly
  2. Stop a training early before k_max is reached if error doesn't improve any more
mc10011 commented 5 years ago

My approach assumes that if a step doesn't improve the Error at least by the min_E_step, then the step is not successful "line 734 if Enew<E and abs(E-Enew)>=min_E_step: ". This way we have the chance to re-adjust the learning rate and get a successful step right after a min_E_step step.

yabata commented 5 years ago

My approach assumes that if a step doesn't improve the Error at least by the min_E_step, then the step is not successful "line 734 if Enew<E and abs(E-Enew)>=min_E_step: ". This way we have the chance to re-adjust the learning rate and get a successful step right after a min_E_step step.

Sure, but should the counter "early" not be reset to 0 if there is a successfull step again? Such that it only brekas if this happens 5 times in a row.

mc10011 commented 5 years ago

Changes done.

PS: Since I introduced this early criterium I have never encountered the "infinite-training" problem again

yabata commented 5 years ago

Thank you for your contribution :)

mc10011 commented 5 years ago

Hello Dennis,

thank you for accepting my request!

Regarding my application, I am using your toolbox for my master thesis in Marine engineering with the target to simulate an internal combustion engine as an RNN and compare it with its linear and first principle thermodynamic MATLAB/Simulink models. Eventually, if all work out well, we plan to add the RNN model as the reference for a Controls strategy.

Thank you for your work and for making it available online.

Best regards, Pantelis

On Mon, 22 Apr 2019 at 20:47, Dennis Atabay notifications@github.com wrote:

Thank you for your contribution :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yabata/pyrenn/pull/8#issuecomment-485491047, or mute the thread https://github.com/notifications/unsubscribe-auth/AGHAD6KNDHUWY2D5FFGB6DDPRX2ZVANCNFSM4GYGSMUA .

yabata commented 5 years ago

Hi Pantelis,

Very interesting topic! I hope it works out! Good luck with your thesis.

Cheers

Dennis

Von: mc10011 notifications@github.com Gesendet: Dienstag, 23. April 2019 08:46 An: yabata/pyrenn pyrenn@noreply.github.com Cc: Dennis Atabay dennis.atabay@gmail.com; State change state_change@noreply.github.com Betreff: Re: [yabata/pyrenn] Update Train_LM with minimum error step criterium (#8)

Hello Dennis,

thank you for accepting my request!

Regarding my application, I am using your toolbox for my master thesis in Marine engineering with the target to simulate an internal combustion engine as an RNN and compare it with its linear and first principle thermodynamic MATLAB/Simulink models. Eventually, if all work out well, we plan to add the RNN model as the reference for a Controls strategy.

Thank you for your work and for making it available online.

Best regards, Pantelis

On Mon, 22 Apr 2019 at 20:47, Dennis Atabay notifications@github.com wrote:

Thank you for your contribution :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yabata/pyrenn/pull/8#issuecomment-485491047, or mute the thread https://github.com/notifications/unsubscribe-auth/AGHAD6KNDHUWY2D5FFGB6DDPRX2ZVANCNFSM4GYGSMUA .

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/yabata/pyrenn/pull/8#issuecomment-485664800 , or mute the thread https://github.com/notifications/unsubscribe-auth/ACGZMJR4OYXXKJLDQEA24NLPR2WB3ANCNFSM4GYGSMUA .