Does vl_nnconv implement 3D convolution or multi-channel 2D convolution?

[jiangwenchao]

I want to implement 3D CNN with the Matconvnet, but I am not sure whether the convolution is 3D convolution or multi-channel 2D convolution? More specially, If I have an image with M_N_K where M is the number of row, N is the number of column and K is number of channel. The filter is with M'_N'_K'. The first way to do convolution is just 3D convolution. The second way is process each channel seperately, which is filtering the first channel with the filter in the first channel, and so on. Which way does the Matconvnet choose? Thank you very much.

[vedaldi]

Hi, the current convolution code implements multi-channel 2D convolution.

On 22 Jun 2015, at 06:15, jiangwenchao notifications@github.com wrote:

I want to implement 3D CNN with the Matconvnet, but I am not sure whether the convolution is 3D convolution or multi-channel 2D convolution? More specially, If I have an image with MNK where M is the number of row, N is the number of column and K is number of channel. The filter is with M'N'K'. The first way to do convolution is just 3D convolution. The second way is process each channel seperately, which is filtering the first channel with the filter in the first channel, and so on. Which way does the Matconvnet choose? Thank you very much.

— Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/177.

[pengsun]

Hi @jiangwenchao, if you're working on medical/volume image (like me) and interested in 3D ConvNet, you may take a look at this: https://github.com/pengsun/MexConv3D

It provides 3D convolution and maxpooling and works well with matconvnet.

[omair-kg]

Hi @pengsun , I integrated your 3dConv and maxpooling3D implementation into matconvnet. Now I need to ask a fairly stupid question. Can you suggest how I can can check if the layers have been correctly implemented. Thanks in anticipation.

[tzczsq]

Hi @omair-kg , Could you tell me how to integrate 3DConv and maxpooling 3D into matconvnet?

[omair-kg]

Hi @tzczsq. I added the 3D conv and pooling layer to the vl_simplenn wrapper since at present i don't need DAG functionality with these layers. You need to define new cases in the simplenn wrapper for both backward and forward passes. You just need to have a look at the assignment of variables according to mex_conv3d and mex_maxpool3d. You would also need to add a 'squeeze' layer to remove the singleton dimension.

[minhlevn]

Hi, I run the example code in MexConv3D, but it s error: Error in test (line 21) res.x1 = mex_conv3d(X,F1,B1,'stride',c_str);

I don't know why mex_conv3d is empty?

[tzczsq]

Hi, sorry, I can not remember this problem, since it takes too long. Shiqing

At 2016-11-27 09:33:15, "minhlevn" notifications@github.com wrote:

Hi, I run the example code in MexConv3D, but it s error: Error in test (line 21) res.x1 = mex_conv3d(X,F1,B1,'stride',c_str);

I don't know why mex_conv3d is empty?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[divined2004]

Hello everybody

I know this is an old thread but I'm still interested in integrating mex_conv3d into MatConvNet. I proceeded according to the advice of omair-kg and inserted a case for mex_conv3d into the switch statement for the forward and backward stages into the vl_simplenn wrapper.

My problem is that I'm not sure how to map the MatConvNet variables into the format that Mex_conv3D accepts. According to MatConvNet the input res(i).x has a size of (W,H,C,B), where W:Width, H:Height, C:Channels, B: batch size. On the contrary mex_conv3D expects a parameter of size (W,H,C,#feature_maps, #instances). Furthermore the filter is in the format weights l.weights{1}, l.weights{2} in MatConvNet but is expected to be in the format (3D volume + #input feature maps + #output feature maps).

How am I supposed to map the initial format of MatConvNet variables into the one that mex_conv3d requires. This is how I'm trying to call mex_conv3d for the forward pass:

case 'mex_conv3d' % pad = [0,0, 0,0, 0,0]; % 3D higher/lower padding % stride = [1,1,1]; % 3D stride res(i+1).x = mex_conv3d(res(i).x, l.weights{1}, l.weights{2}) ;

but I get the following error:

Error using mex_conv3d

feature maps of F and X should match: size(F,4)==size(X,4).

Could someone that successfully integrated mex_conv3d and/or mex_pool3d give me some advice or point me to some readily available code?

[divined2004]

For anyone still interested, this is how I integrated mex_conv3d into the forward and backward phases of MatConvNet:

Forward pass;

  case 'mex_conv3d'
    pad    = [0,0, 0,0, 0,0]; % 3D higher/lower padding
    stride = [1,1,1];         % 3D stride
    size1 = size(res(i).x);   % get size of res(i).x (input to layer)
    if (gpuMode)
      szX = [size1(1) size1(2) size1(3) l.dim1(4) net.meta.trainOpts.batchSize];
      X = gpuArray.rand(szX, 'single');
      X(:,:,:,:,:) = res(i).x;
     szF = [l.dim1(1), l.dim1(2), l.dim1(3), l.dim1(4), l.dim1(5)];
     F = gpuArray.rand(szF, 'single');
     F(:,:,:,:,:) = l.weights{1};
     szB = size(l.weights{2});
     B = gpuArray.rand(szB, 'single');
     B(:,:) = l.weights{2};

     %% fprop
     Y = mex_conv3d(X,F,B, 'pad', pad, 'stride',stride); % 3d-conv
     res(i+1).x = Y;
    else
      X = single(zeros(size1(1), size1(2), size1(3), l.dim1(4), net.meta.trainOpts.batchSize));  % dummy   array same size as input but with an additional dimension
      X(:,:,:,:,:) = res(i).x;   % copy input image into new array as expected by mex_conv3d
      F = single(zeros(l.dim1(1), l.dim1(2), l.dim1(3), l.dim1(4), l.dim1(5)));
      F(:,:,:,:,:) = l.weights{1};
      B = l.weights{2};

      %% fprop
      Y = mex_conv3d(X,F,B, 'pad', pad, 'stride',stride); % 3d-conv
      res(i+1).x = Y;

    end

The backward pass is much the same but with the following changes on the last three lines:

   %% bprop
    [res(i).dzdx, dzdw{1}, dzdw{2}] = mex_conv3d(X,F,B, res(i+1).dzdx, 'pad', pad, 'stride', stride); % 3d-conv

The l.dim1 is a 5-D matrix that contains the filter dimensions. Hope this helps people wishing to integrate mexconv3d into MatConvNet.

[mafzal1]

@divined2004 can you please share codes for incorporating mex_maxpool3d in vl_simplenn too, if you have them?