akrizhevsky / cuda-convnet2

Automatically exported from code.google.com/p/cuda-convnet2
Apache License 2.0
783 stars 290 forks source link

Multiple data layer with binomialcrossEntropyCostLayer #8

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. There are more than 2 data layers
2. Use BinomialCrossEntropyCostLayer cost layer
3.

What is the expected output? What do you see instead?

The (output) dimension of third data layer is 1700.

Without adding "start=0 end=1700" in the layer definition file for the third 
layer, the program will crash, 
 The error info is

src/../../cudaconv3/include/../../nvmatrix/include/nvmatrix.cuh:376: void 
NVMatrix::applyBinary(Op, NVMatrix&, NVMatrix&, CUstream_st*) [with Op = 
BinomialCrossEntOperator]: Assertion `this->isSameDims(b)' failed.

Then I add the following lines in layer.cu
          int numCases = labels.getLeadingDim(); //line 2108 in layer.cu
          printf("%d %d=====\n\n",probs.getNumRows(), probs.getNumCols()); 
          printf("%d %d=====\n\n",labels.getNumRows(),labels.getNumCols());

     The size of labels is (0, 1024), and the size of probs is (1700,1024).

After adding start=0 end=1700, the size will be correct, but I got the 
following error, 
        CUDA error at src/../include/memory.cuh:272 code=2(cudaErrorMemoryAllocation) "cudaMalloc(data, size)" 

What version of the product are you using? On what operating system?
Cuda5.5, CentOS6.5

Please provide any additional information below.

Original issue reported on code.google.com by itsup...@gmail.com on 26 Aug 2014 at 3:19

GoogleCodeExporter commented 9 years ago
cudaMalloc problem is caused by large batch size.
With start=0 end=1700 in the layer definition file for the third data layer, it 
works fine. 

Original comment by itsup...@gmail.com on 16 Nov 2014 at 2:04