search

joncox123 / Cortexsys

Matlab GPU Accelerated Deep Learning Toolbox
Other
70 stars 28 forks source link

Learning Rate for Gradientdescent #10

Open GraboDan opened 7 years ago

GraboDan commented 7 years ago

Hi Jon, When I use the normal gradientdescent function and I set alpha to zero (as your comment in that line implies that this will lead to a variable learning rate) the learning rate will actually stay at 0 the entire time and my error for the MNIST stays at ~90%. If I set alpha to a value it will use this as the first value and then decay it according to the equation in alphatau. So it works, but the comment is misleading. There seems to be only the option of using a variable learning rate. A constant seems not implemented.

Anyways, great code :)

RMano82 commented 7 years ago

Hi @GraboDan . I did try to use this toolbox but with no good luck. Can you please share a working and converging example. Thanks a lot