How to Speed-up training?

[AminSuzani]

Hi,

Thanks for your great package. I am training a network with three hidden layers, 400 features, and 78 targets. I use the code below for training. The result on test data is impressive, but it takes too much time to train (about two days). Is there a way to parallelize training on multiple cores or GPU? Or any other suggestions to speed-up the training process?

train_set = [X_minmax, Y_xyz_minmax] e = theanets.Experiment(theanets.feedforward.Regressor, layers=(featuresNum, 200, 300, 150, vertebNum*3), optimize= 'layerwise' , activation='tanh', ) e.run(train_set, train_set)

[lmjohns3]

The theanets package uses theano to perform its computations. Have you enabled theano to run on your GPU? It's pretty easy; just prefix your Python script on the command-line with:

THEANO_FLAGS='floatX=float32,device=gpu0' python my-theanets-script.py

There are a few other configuration options worth setting; see http://deeplearning.net/software/theano/library/config.html for more information.

There's also an open issue for theanets to be able to distribute datasets across an ipython cluster. I haven't had a chance to work on it, but hope to do that in the next few weeks.

[lmjohns3]

Also, the layerwise pretrainer can be pretty expensive, since it effectively trains one model for each hidden layer in your network.

[lmjohns3]

I'm going to go ahead and close this, it seems to have been resolved. Please reopen if that's not the case.