search

soumith / cuda-convnet2.torch

Torch7 bindings for cuda-convnet2 kernels!
Apache License 2.0
40 stars 16 forks source link

Bindings for SpatialMaxPooling #13

Closed szagoruyko closed 10 years ago

szagoruyko commented 10 years ago

It seems that it is slower than cunn version, I don't know why.

soumith commented 10 years ago

awesome! How much slower is it? Is it significantly slower or just slightly...

szagoruyko commented 10 years ago

@soumith when I run tests the ratio is different, sometimes 0.5, sometimes 2.0, even with big nloop. But I guess comparing to SpatialConvolution this difference is a drop in an ocean.

soumith commented 10 years ago

yes correct, also, the tests dont actively reflect the speedup that alex optimized for (to test speedup, you have to use batchsize 128 and x*32 output feature maps)

szagoruyko commented 10 years ago

@soumith ok, got it. by the way, do you know if other modules are going to be ready soon? I plan to work on normalizations and locally connected layers

soumith commented 10 years ago

unfortunately i am in a grey area right now, as i just switched workplaces, so at least for the next few weeks i'm not sure if i can work on this. there is another fork by @adampolyak where he seems to have implemented local unshared conv layer, maybe you can get in touch with him?

szagoruyko commented 10 years ago

@soumith I will try, thanks!

adampolyak commented 10 years ago

I finished implementing the local unshared conv layer. I will pull request it once I'll finish testing it.

adampolyak commented 10 years ago

By the way, maybe you guys have an idea for a way to check LocalSpatialConvolution? Torch doesn't implement it.

szagoruyko commented 10 years ago

@adampolyak have you tried nn.Jacobian? I also have code for unshared forward (not batch) convolution on CPU

szagoruyko commented 10 years ago

@adampolyak it is here: https://github.com/szagoruyko/nn I haven't finished it. named SpatialConvolutionLocal, only forward works.

adampolyak commented 10 years ago

I tried using nn.Jacobian but apparently it doesn't work well with CudaTensor (cunn doesn't have Jacobian tests). It works only when using with DoubleTensors.

When running Jacboian tests I get the following err for LocalConvolutionLatyer: LT(<) violation _val=0.11865755164763_, condition=1e-05

I'll test the forwarding code using your nn branch, any ideas on how to check back propagation?

p.s.

  1. ccn2.SpatialConvolution has the same error when using Jacobian: LT(<) violation _val=0.11551479995251_, condition=1e-05
  2. I added tests/jac_test.lua in my branch that checks these errors.
szagoruyko commented 10 years ago

@adampolyak I got the same problem with SpatialMaxPooling, condition 1e-05 is violated, val = 0.01 Given that ccn2.SpatialConvolution outputs are tested to be equal to cunn.SpatialConvolutionCUDA outputs and such error is normal for cunn, it should be normal for ccn2 also Can you do a quick test it for the issue reported here: https://github.com/soumith/cuda-convnet2.torch/issues/10 ?

soumith commented 10 years ago

fixed the jacobian tests. https://github.com/soumith/cuda-convnet2.torch/blob/master/test/test.lua#L203