search

pluskid / Mocha.jl

Deep Learning framework for Julia
Other
1.29k stars 254 forks source link

convolutional autoencoders #53

Open lelayf opened 9 years ago

lelayf commented 9 years ago

I would find it useful to have an example of convolutional auto-encoder. Would a tied max un-pooling layer be necessary, using switches from its twin pooling layer to keep track of argmax'es (as in Zeiler's data viz paper)? And possibly a tied deconv layer ? How would layer-wise greedy training be performed to initialize a deep CNN such as the one used for CIFAR-10 ?

pluskid commented 9 years ago

@lelayf We have a deconv layer but that is currently only implemented for CPU backend, so I have not merged it into master yet. But we did not have a un-pooling layer yet. And the weights for the deconv layer cannot be tied yet. Could you point me to the paper that use layer-wise pretraining for CNN on CIFAR-10? Thanks!

lelayf commented 9 years ago

I haven't seen a paper demonstrating this on CIFAR-10 per-se, the paper I was referring to illustrate memoizing the argmax in pooling layers to be able to reuse those locations when un-pooling. The paper is "Visualizing and Understanding Convolutional Networks" http://arxiv.org/pdf/1311.2901v3.pdf