search

IDSIA / brainstorm

Fast, flexible and fun neural networks.
Other
1.3k stars 152 forks source link

Implement StopAfterThresholdReached hook #92

Closed iver56 closed 8 years ago

iver56 commented 8 years ago

This hook stops the training when a certain accuracy, for example 97 %, is reached. Since it might be useful for others, I thought I'd share it.

Example usage:

trainer.add_hook(bs.hooks.StopAfterThresholdReached('validation.Accuracy',
                                                    threshold=0.97,
                                                    criterion='at_least'))

This needs a test and some documentation. If there would be documentation and tests for the existing EarlyStopper, then I could write similar docs and tests for this hook.

flukeskywalker commented 8 years ago

Thanks! Here are my thoughts:

  1. The behavior for 'min' and 'max' criteria is contrary to EarlyStopper, and I believe incorrect.
  2. 'at_least' and 'at_most' seem confusing (the second one more so). It's probably best to simply use 'min' and 'max', which should be easier to explain in the docs: The hook stops training if the minimum/maximum value reaches the threshold.
  3. threshold should not have a default value.
  4. Can you please add some doc and example usages? :)
iver56 commented 8 years ago

I can fix those things. Re "at_least": My thinking went something like "stop when accuracy is at least 97 %".

When it comes to documentation, I think I'll wait until there's documentation with examples for EarlyStopper. Then I can copy, paste and adapt so the documentation style becomes consistent.

flukeskywalker commented 8 years ago

Yes, 'at least' parses fine, but saying stop when loss is at most 0.1 doesn't really work well.

landscape-bot commented 8 years ago

Code Health Repository health decreased by 0.02% when pulling 286a20b on iver56:threshold-hook into 8ab60fa on IDSIA:master.

landscape-bot commented 8 years ago

Code Health Repository health decreased by 0.02% when pulling d2a92f4 on iver56:threshold-hook into f80250c on IDSIA:master.

landscape-bot commented 8 years ago

Code Health Repository health decreased by 0.04% when pulling 404f97a on iver56:threshold-hook into f80250c on IDSIA:master.

iver56 commented 8 years ago

@flukeskywalker I've fixed the things you mentioned

landscape-bot commented 8 years ago

Code Health Repository health decreased by 0.04% when pulling 81be287 on iver56:threshold-hook into f80250c on IDSIA:master.

flukeskywalker commented 8 years ago

Thanks!