Large Minibatch Sizes Crash Application

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Load Gaussian IDX data -- 800 luminance images of size 128x128
2. Set minibatch size to greater than 12
3. Press 'start'

What is the expected output? What do you see instead?
Expect training to commence.  Instead, application crashes with, "VisualRBM has 
stopped working."

What version of the product are you using? On what operating system?
Windows binaries from 2012/09/04.  Running Windows 7 with most recent CUDA 
drivers as of 2012/09/06 (version 320?  302?)  Running on an nVidia 550 Ti.

Please provide any additional information below.

Original issue reported on code.google.com by jo.j...@gmail.com on 7 Sep 2012 at 1:13

Attachments:

• training_data.idx
• settings.vrbmparameters
• DxDiag.txt

[GoogleCodeExporter]

Discovered the issue only seems to happen on the first or second training 
rounds.  After training once with minibatch size of 10, can bump to 50 or 100 
without crashing.

Original comment by jo.j...@gmail.com on 7 Sep 2012 at 1:10

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Found the problem, I was using Minibatch count rather than Minibatch size in 
the RBMProcessor (for copying textures back from GPU to frontend).  So when the 
number of Minibatches became less than the Minibatch size, you'd get an index 
out of bounds and crash.  With MNIST the issue never came up since I would have 
around 6000 minibatches of size 10.  Things should be a bit faster now though 
for large datasets, since I'm not memcpying around garbage memory.

Original comment by pospes...@gmail.com on 9 Sep 2012 at 4:37

• Changed state: Fixed