nitbix
commented
11 years ago Hi Alex,
I ha e a change set in review that includes ensemble techniques, and I'm using that code for a review paper I'm working on. One of the things I wanted to include late during the process was dropout. Maybe we can pool resources and work on it together? I would be more than happy to help! If you want to see what I did for ensembles look at my submission (there's probably some cleanup still to do, but the juice is mostly there).
Best, Alan On 29 Jan 2013 16:22, "Alex Robbins" notifications@github.com wrote:
Dropout is the concept of training a neural network whose hidden layer neurons have only a .5 probability of being present in any training run. The end result is that you are effectively training an ensemble of neural networks that have massive weight sharing. This speeds training greatly, and eliminates some of the overfitting often associated with neural nets. For instance, neural nets with dropout never need to have their training stopped early to avoid overfitting.
Discussed in this paper: http://arxiv.org/abs/1207.0580 And in this Google tech talk: http://www.youtube.com/watch?v=DleXA5ADG78
I'd be willing to contribute code if this is something you'd be willing to include. Thanks!
— Reply to this email directly or view it on GitHubhttps://github.com/encog/encog-java-core/issues/130.
alexrobbins
jeffheaton
challarao
thorinii
Dropout is the concept of training a neural network whose hidden layer neurons have only a .5 probability of being present in any training run. The end result is that you are effectively training an ensemble of neural networks that have massive weight sharing. This speeds training greatly (relative to training the entire ensemble separately), and eliminates some of the overfitting often associated with neural nets. For instance, neural nets with dropout never need to have their training stopped early to avoid overfitting.
Discussed in this paper: http://arxiv.org/abs/1207.0580 And in this Google tech talk: http://www.youtube.com/watch?v=DleXA5ADG78
I'd be willing to contribute code if this is something you'd be willing to include. Thanks!