hughperkins / DeepCL

OpenCL library to train deep convolutional neural networks
Mozilla Public License 2.0
867 stars 199 forks source link

Add momentum to SGD trainer #9

Closed hughperkins closed 9 years ago

hughperkins commented 9 years ago
hughperkins commented 9 years ago

Started work on this, in 'momentum' branch. https://github.com/hughperkins/DeepCL/tree/momentum

hughperkins commented 9 years ago

So, I'm thinking about how this would work / be used.

Maybe something like, on the commandline one has an option like trainer=sgd, which at the moment would have the only one possible option, ie sgd, but in the future could change.

Then, depending on which trainer is chosen, additional options could be available, eg momentum, learningrate etc, and these would be handed to the sgd somehow.

I seem to have created a layer-specific sgd class, that has to be attached to each conv and fc layer, twice actually, once for biases, and once for non-biases, but this can be done simply by the commandline parser, ie:

  1. set up the network
  2. add appropriate trainer to each weighted layer
hughperkins commented 9 years ago

Done :-) In master branch.