Weather Dataset prediction always FALSE

[flipdazed]

Issue The extension to the weather data is a boolean classification task. The affected models are both only returning False for all entries.

Affected

train_mlp.py
train_db.py

Reproduction

./predict.py -m dump/models/result_dbn.pkl -s 0,-1,10
./predict.py -m dump/models/result_mlp.pkl -s 0,-1,10

Runlog (open in OS X terminal with cat runlog.txt to enable full colour output)

• DBN runlog

[flipdazed]

RBM not learning data set Bad hyper-parameters or related to the cost issue of #2 ?

nodes Here I tried [15000, 15000, 15000]. I assumed I would need >10000 nodes in at least the first RBM layer to model cross-relationships between the data sets. I made this assumption from reading this paper by Geoffrey Hinton who seemed to arbitrarily just bump up the first number of nodes depending on the complexity of the task at hand. I’m not entirely sure how complex this particular task is though and how many nodes to include and how to distribute them.

learning rate, epochs, batch size, early stopping I don’t think this is the problem here as these would just cause the learning to be slower/faster rather than the wrong direction. Mini-batch size seems reasonable at 10 for a total sample size of 582. Early stopping is not part of the pre-training so can’t be responsible for the direction issues

k, number of Gibbs steps in CD/PCD Is this possibly responsible? 1 is the lowest this could possibly be!

[flipdazed]

MLP

Weights

A sample of the weights showing activity early on then nothing for the rest of the run time

[flipdazed]

closed as too general.