search

terkkila / cgml

Machine Learning with Computational Graphs
Apache License 2.0
2 stars 0 forks source link

Unsupervised pre-training #21

Open terkkila opened 10 years ago

terkkila commented 10 years ago

Instead of randomly initializing the weight matrices, we run autoencoder pre-training before learning the classifier/regressor: http://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf

terkkila commented 10 years ago

This requires that we can turn any Computational Graph into an Autoencoder. One solution is to take the CG, discard the output layer, mirror the other layers with transposed weights, learn the autoencoder, and revert the CG back to original one but with the weight values kept as they are.