BVLC / caffe

Caffe: a fast open framework for deep learning.

http://caffe.berkeleyvision.org/

Other

33.97k stars 18.72k forks source link

RBM layer ? (+DBN) #1207

Closed cbalint13 closed 9 years ago

cbalint13 commented 9 years ago

Anyone thinking or planning to implement $subj ? I think it would be quite usefull to have:

RBM, GRBM with CD-k, PCD-k

shelhamer commented 9 years ago

Contributing an RBM layer and CD-k solver would be a fine start. I know there is interest in probabilistic models and RBMs in Caffe from conversations at conferences and the like so it's a worthy project for the interested researcher / developer.

274 is an early but cancelled attempt at an RBM layer. At the time I suggested a hybrid solution in Caffe + Theano, at least for the purposes of prototyping, but have since changed my mind and agree that having it all in Caffe would be helpful and interesting. One might train models with discriminative and generative components if it's all in the same framework.

elador commented 9 years ago

If there is no RBM layer yet in Caffe, what is this commit then if I may ask? I am a bit confused.

https://github.com/BVLC/caffe/commit/7f43120be92c2a70991d81be0f80a763354e2944

kloudkl commented 9 years ago

Don't think it works at all. Either write one from scratch, or use Pylearn2.

cbalint13 commented 9 years ago

@shelhamer ,

I am willing to contribute with RBM layer. My first step would be:

Use this very clear example: https://github.com/cudamat/cudamat/blob/master/examples/rbm_cudamat.py
Implement in caffe's way of API, add very simple MNIST example with some visualisation.

If all fine in a very future next stage would do:

Use deepnet as reference: https://github.com/nitishsrivastava/deepnet
Add l1, l2 constraints
Add sparsity constraints
Add gaussian-rbm (linear input)
Add CD-k, and PCD
More examples with CIFAR-10, ReLU within GRBM samples and many more.
More multimodal examples

You can close this Issue, will post a PR (asap) instead.

kashefy commented 8 years ago

@cbalint13 Did you end up creating a PR for this? still planning to? dropped it?

krikru commented 8 years ago

+1 on @kashefy's comment. Why was this issue closed?

cbalint13 commented 8 years ago

@krikru ,

An effort is going on: https://github.com/BVLC/caffe/pull/3568 . My time is limited but this on my top todo list and will try enhance with more features.

krikru commented 8 years ago

@cbalint13 Sweet. I was just curious :)