search

nicholas-leonard / dp

A deep learning library for streamlining research and development using the Torch7 distribution.
Other
343 stars 140 forks source link

Early stopping when accuracy reaches 1 consistently #161

Closed eulerreich closed 8 years ago

eulerreich commented 9 years ago

Is it currently possible to do this? If not, any suggestions on how to implement it?

nicholas-leonard commented 9 years ago

What do you mean by when accuracy reaches 1? Do you mean when it reaches 100%? What about the current early-stopping doesn't work for you?

eulerreich commented 9 years ago

Yea when it reaches 100%. I want the experiment to stop only when accuracy reaches 100% (or stopped manually by me). Right now I can set the maxTries to 99999 but then it wouldn't stop at all when accuracy reaches 100%.

nicholas-leonard commented 9 years ago

did you use EarlyStopper argument maximize = true and this : https://github.com/nicholas-leonard/dp/blob/master/examples/convolutionneuralnetwork.lua#L228?

eulerreich commented 9 years ago

yea.

On Mon, Sep 21, 2015 at 3:19 PM, Nicholas Léonard notifications@github.com wrote:

did you use EarlyStopper argument maximize = true and this : https://github.com/nicholas-leonard/dp/blob/master/examples/convolutionneuralnetwork.lua#L228 ?

— Reply to this email directly or view it on GitHub https://github.com/nicholas-leonard/dp/issues/161#issuecomment-142097484 .

nicholas-leonard commented 9 years ago

Oh that is because you set maxTries to high. You should set it to something like 50 or 100. There is currently no way to tell it to stop when it reaches the maximum accuracy. You could however submit a Pull Request to modify EarlyStopper to have a maxima argument.

eulerreich commented 9 years ago

What I would like is something along the lines of some arguments stop_threshold=1-epsilon, stop_threshold_period=50 which would mean to stop after reaching accuracy 1 - epsilon for 50 consecutive minibatches.

On Mon, Sep 21, 2015 at 3:20 PM, Greg Yang sorcererofdm@gmail.com wrote:

yea.

On Mon, Sep 21, 2015 at 3:19 PM, Nicholas Léonard < notifications@github.com> wrote:

did you use EarlyStopper argument maximize = true and this : https://github.com/nicholas-leonard/dp/blob/master/examples/convolutionneuralnetwork.lua#L228 ?

— Reply to this email directly or view it on GitHub https://github.com/nicholas-leonard/dp/issues/161#issuecomment-142097484 .

nicholas-leonard commented 9 years ago

Hmmm. In that case, setting maxTries = 50 will do the trick for you.

eulerreich commented 9 years ago

The thing is that I don't want the experiment to stop after 50 minibatches without surpassing the previous max IF IT DOESN'T SURPASS THE THRESHOLD. It may very well be that the experiment will never reach accuracy 1, in which I want it to keep running until I stop it myself.

On Mon, Sep 21, 2015 at 3:39 PM, Nicholas Léonard notifications@github.com wrote:

Hmmm. In that case, setting maxTries = 50 will do the trick for you.

— Reply to this email directly or view it on GitHub https://github.com/nicholas-leonard/dp/issues/161#issuecomment-142101994 .

nicholas-leonard commented 9 years ago

Yeah I see what you mean.