search

facebookresearch / SparseConvNet

Submanifold sparse convolutional networks
https://github.com/facebookresearch/SparseConvNet
Other
2.04k stars 332 forks source link

Implementation Details of SparseConvNet for CIFAR-10 Dataset #29

Closed leobxpan closed 6 years ago

leobxpan commented 6 years ago

Hi Benjamin,

I am very interested in your paper "Spatially-sparse convolutional neural networks", which was published in 2014, and I am currently re-implementing your network for CIFAR-10 (I am currently implementing the "vanilla" sparse convnet without using fractional max pooling). While I have encountered several implementation problems, and since I failed to find a valid email address of you, I therefore posted my questions here. I am sorry for the inconvenience.

  1. In your paper you said that you designed a DeepCNiN (5, 300) network for this CIFAR-10 problem, which should have the following architecture:
Input - 300C3 - MP2 - 300C1 - 
          - 600C2 - MP2 - 600C1 -
          - 900C2 - MP2 - 900C1 - 
          - 1200C2- MP2 - 1200C1 -
          - 1500C2- MP2 - 1500C1 - 
          - 1800C2- 1800C1 - Softmax - Output

But in your code posted here, I found that you actually called addLeNetLayerROFMP (which first adds a Conv layer, then a NiN layer and then a ROFMP layer, whose order is different from that proposed in your paper) for 12 times and then addLeNetLayerMP for 2 times, finally a Softmax layer (the number of layers is also different). Since this architecture is different from the one you proposed in your paper, I wonder which one did you finally adopt and can yield the best result (or the result posted in your paper)?

  1. I wonder what is the initialization method that you used for the weight and bias (e.g. Gaussian distribution)? I didn't really find your code for this.

Thank you for your patience and help.

btgraham commented 6 years ago

https://github.com/btgraham/SparseConvNet/blob/master/SparseConvNet/cifar10.cpp relates to https://arxiv.org/abs/1412.6071 which is not currently implemented in this repo.

If you are interested in CIFAR-10 I recommend you start with something like: https://github.com/liuzhuang13/DenseNet

Regards Ben

fengxiuyaun commented 6 years ago

@btgraham Hi, I am very interested in your paper "Spatially-sparse convolutional neural networks",too. But I don't know how to implement the network that trains Imagenet data by using “ facebookresearch/SparseConvNet”, beacuse of the fact how images are as the input of networks.