soumith / cuda-convnet2.torch

Torch7 bindings for cuda-convnet2 kernels!
Apache License 2.0
40 stars 16 forks source link

cuda convnet2 insisting on #filters to be divisible by 32 #17

Closed adampolyak closed 10 years ago

adampolyak commented 10 years ago

The following assert exists in cuda-convent2: https://github.com/soumith/cuda-convnet2.torch/blob/master/cudaconv3/src/img_acts.cu#L1208

This causes failure in some cases that cuda-convnet2 should support, for example:

model = nn.Sequential()
model:add(ccn2.SpatialConvolutionLocal(16, 16, 63, 9))
model:add(nn.ReLU())
model:backward(torch.rand(16,63,63,128))

will fail, because the number of filters is 16 which doesn't pass the assert check. However, the documentation here mentions that this number should be a multiple of 16.

  1. Is this check really required?
  2. In some cases, such as this, the flow of the code uses a filter cache size of 16: https://github.com/soumith/cuda-convnet2.torch/blob/master/cudaconv3/src/img_acts.cu#L1345 does this mean that removing the assert will make the code above work correctly?
soumith commented 10 years ago

(1) The comment right by the check says there's a bug in the calling code blow, i haven't really looked into it (the comment is from Alex) (2) I am not sure

I think you can try removing the assert (change the assert to %16) and run 1000 iterations of the unit tests with batch-size as a multiple of 16, and that might uncover any issues. The unit tests are pretty robust and will uncover any issues.