faster_rcnn on k-channel image dataset

[mtrth]

I followed the installation steps and also downloaded the models but when I run the demo script_faster_rcnn_demo.m , I get the error:

fast_rcnn startup done GPU 1: free memory 5307551744 GPU 2: free memory 6312861696 Use GPU 2

Reference to non-existent field 'bottom_id_vecs'.

Error in caffe.Net (line 74) self.attributes.bottom_id_vecs = cellfun(@(x) x+1, self.attributes.bottom_id_vecs, 'UniformOutput', false);

Error in caffe.get_net (line 28) net = caffe.Net(hNet);

Error in caffe.Net (line 33) self = caffe.get_net(varargin{:});

Error in script_faster_rcnn_demo (line 33) rpn_net = caffe.Net(proposal_detection_model.proposal_net_def, 'test');

[ShaoqingRen]

@mtrth

The "bottom_id_vecs" is initalized in Line 73--77, caffe.Net, as % expose bottom_id_vecs and top_id_vecs for public read access self.attributes.bottom_id_vecs = cellfun(@(x) x+1, self.attributes.bottom_id_vecs, 'UniformOutput', false); self.bottom_id_vecs = self.attributes.bottom_id_vecs; self.attributes.top_id_vecs = cellfun(@(x) x+1, self.attributes.top_id_vecs, 'UniformOutput', false); self.top_id_vecs = self.attributes.top_id_vecs;

Please make sure that the code of caffe.Net is correct.

[mtrth]

Hi, thanks for the suggestion it worked, now I am able to run the demo. I had a few questions though,

1) Can you give me a few pointers of how to train faster_rcnn on my own data? 2) Also can faster_rcnn be used on k-channel dataset (I have 5 channel image dataset) or is limited to RGB ?

Thanks

[ShaoqingRen]

@mtrth 1) for your own data, you need to set up your imdb and roidb as we do for VOC dataset. 2) this code is not limited to RGB. If you need more input channel (e.g, 5), just update the prototxt's input to 5 channel and feed into 5 channel images will work.

[mtrth]

My dataset has 2 classes; with 1000 training images of (5,256,256) also corresponding ground truth data (1,256,256) which is a binary image either 0 or 1 to represent the 2 classes.

I use this python script to create my lmdb of ROI's previously I had 1 label for each image which I passed to the variable y. But I want to use the ground truth mask as in VOC dataset, so the y now is a array (1,256,256); should I create a separate lmdb for ground truth or how exactly should I pass the ground truth data ?

import numpy as np import lmdb import caffe

X = imread('image0001.tif') # read N image y = 1 # image label map_size = X.nbytes * 10

env = lmdb.open('mylmdb', map_size=map_size)

with env.begin(write=True) as txn:

txn is a Transaction object

for i in range(N):
    datum = caffe.proto.caffe_pb2.Datum()
    datum.channels = X.shape[1]
    datum.height = X.shape[2]
    datum.width = X.shape[3]
    datum.data = X[i].tobytes()  # or .tostring() if numpy < 1.9
    datum.label = int(y[i])
    str_id = '{:08}'.format(i)
    with env.begin(write=True) as txn:
      txn.put(str_id.encode('ascii'), datum.SerializeToString())

[ShaoqingRen]

@mtrth In my understanding, your image is 5 in channel and 256_256 in width_height. And the output you need is a mask with 256*256. Then I think the task you want to do is semantic segmentation ? This code is designed for detection. If you want to do instance aware segmentation, you can modified the code to support it. Otherwise, I think fully convoluation network (FCN) maybe is more suitable for you.

[vessavana]

@ShaoqingRen i have got the same problem and i'm sure the code of Net.m is correct. i run the demo in CUDA7.5 and caffe is the latest version Would you mind telling me how to solve this problem? Thank u:-)

[abbapst]

@ShaoqingRen I am getting the exact same error message as OP. The code that you posted matches the code in my Net.m file. I am using CPU only with a Mex that I compiled from the official Windows Caffe (your fork did not compile for me). I am using Matlab 2014a (invalid Mex in 2016a). Can you shed any light on what could be going wrong? Thanks.

Edit: After further investigation I believe the error that is leading to "non-existent field" self.attributes.bottom_id_vecs must originate in line 42 of Net.m:

self.attributes = caffe_('net_get_attr', self.hNet_self);

This is a problem with the Mex. I tried replacing it with the original precompiled Mex you provided to make sure it wasn't a problem with my compilation, and I get the same error.

[heyicai]

hi, I have also meet this question: "Reference to non-existent field 'bottom_id_vecs'." And the code is download from website. can you tell how to solve this problem, thank you very much！

[loukbabi]

Hello. I have also encountered the same problem. I am using Ubuntu 16.04, CUDA 8, MATLAB 2016b and I had to use the current branch of Caffe since I was not able to build the "Caffe for Faster R-CNN" repo.

As abbapst pointed out, it seems that the bottom_idvecs field should be created in line 42 along with the rest of the attributes. I additionally did some comparison between the "Caffe for Faster R-CNN" repo and the current Caffe repo from https://github.com/BVLC/caffe. The difference is that caffe.cpp of the current branch includes the following net_get_attr function:

static void net_get_attr(MEX_ARGS) { mxCHECK(nrhs == 1 && mxIsStruct(prhs[0]), "Usage: caffe_('net_get_attr', hNet)"); Net<float>* net = handle_to_ptr<Net<float> >(prhs[0]); const int net_attr_num = 6; const char* net_attrs[net_attr_num] = { "hLayer_layers", "hBlob_blobs", "input_blob_indices", "output_blob_indices", "layer_names", "blob_names"}; mxArray* mx_net_attr = mxCreateStructMatrix(1, 1, net_attr_num, net_attrs); mxSetField(mx_net_attr, 0, "hLayer_layers", ptr_vec_to_handle_vec<Layer<float> >(net->layers())); mxSetField(mx_net_attr, 0, "hBlob_blobs", ptr_vec_to_handle_vec<Blob<float> >(net->blobs())); mxSetField(mx_net_attr, 0, "input_blob_indices", int_vec_to_mx_vec(net->input_blob_indices())); mxSetField(mx_net_attr, 0, "output_blob_indices", int_vec_to_mx_vec(net->output_blob_indices())); mxSetField(mx_net_attr, 0, "layer_names", str_vec_to_mx_strcell(net->layer_names())); mxSetField(mx_net_attr, 0, "blob_names", str_vec_to_mx_strcell(net->blob_names())); plhs[0] = mx_net_attr; }

while the caffe_.cpp of the "Caffe for Faster R-CNN" :

// Usage: caffe_('net_get_attr', hNet) static void net_get_attr(MEX_ARGS) { mxCHECK(nrhs == 1 && mxIsStruct(prhs[0]), "Usage: caffe_('net_get_attr', hNet)"); Net<float>* net = handle_to_ptr<Net<float> >(prhs[0]); const int net_attr_num = 8; const char* net_attrs[net_attr_num] = { "hLayer_layers", "hBlob_blobs", "input_blob_indices", "output_blob_indices", "layer_names", "blob_names", "bottom_id_vecs", "top_id_vecs" }; mxArray* mx_net_attr = mxCreateStructMatrix(1, 1, net_attr_num, net_attrs); mxSetField(mx_net_attr, 0, "hLayer_layers", ptr_vec_to_handle_vec<Layer<float> >(net->layers())); mxSetField(mx_net_attr, 0, "hBlob_blobs", ptr_vec_to_handle_vec<Blob<float> >(net->blobs())); mxSetField(mx_net_attr, 0, "input_blob_indices", int_vec_to_mx_vec(net->input_blob_indices())); mxSetField(mx_net_attr, 0, "output_blob_indices", int_vec_to_mx_vec(net->output_blob_indices())); mxSetField(mx_net_attr, 0, "layer_names", str_vec_to_mx_strcell(net->layer_names())); mxSetField(mx_net_attr, 0, "blob_names", str_vec_to_mx_strcell(net->blob_names())); mxSetField(mx_net_attr, 0, "bottom_id_vecs", int_vec_vec_to_mx_cell_vec(net->bottom_id_vecs())); mxSetField(mx_net_attr, 0, "top_id_vecs", int_vec_vec_to_mx_cell_vec(net->top_id_vecs())); plhs[0] = mx_net_attr; }

It is obvious that this is the main reason of the error but I am not sure if changing only this particular function is going to solve the problem. I am going to try this fix soon and report back but I have little-to-none experience with Matlab mex wrapping and the Caffe library. So I would be grateful for any kind of help.

@ShaoqingRen do you have any feedback? The error is definitely produced because we are not using your version of Caffe. Though, at least for my case, I am aiming for an up-to-date version of Caffe (compatible with the latest libraries) working on Ubuntu. Thank you.

Edit 1: After "some" effort, I found that the above is only one of the things that are needed to be changed. In fact I ended up trying to merge the FasterRCNN-parts of this repo to the current version of Caffe. I think I am very close now, but I encountered some issues with Matlab's c functions called inside the caffe_.cpp . More specifically, I am getting an "undefined reference" error for the functions mxIsGPUArray'mxInitGPU', mxGPUCreateFromMxArray',mxGPUGetDataReadOnly', mxGPUGetNumberOfElements',mxIsGPUArray', `mxGPUDestroyGPUArray'. It seems that for some reason my Matlab does not find the implementation of the above functions. I am using a relatively new version of Matlab (R2016b on Ubuntu 16.04) but I do not think that those functions are removed. I will come back to the post if I have something new. Once I will manage to solve this, I will upload my configuration somewhere. If anyone has any ideas regarding the above error, please let me know. Cheers!

[mra83]

@loukbabi Hi, I encountered the same error, even after modifying caffe_.cpp still it does not recognize 'bottom_id_vecs'. I am trying in a CPU mode on windows. Could you please let me know if you have any update? Thank you!

[loukbabi]

@mra83 Hello there. Since my effort to solve this problem goes a while back, I do not remember the exact steps that I followed in order to have a working repository. What I basically did was tracing every error occurred during compilation and changing the corresponding parts of the code from the official Caffe repo so as to match the "Caffe for Faster R-CNN" branch of this repo. Many things were needed to be changed in many files in order to have the code compiling and frankly I cannot be sure that my changes didn't affect the implementation.

In the end of the day I ended up using the "Caffe for Faster R-CNN" branch included in this repository. The main reason that I was not able to use it in the first place was the incompatibility issues I introduced when I decided to go with Ubuntu 16.04. So my best advice is to use the "Caffe for Faster R-CNN" branch with Ubuntu 14.04 and Matlab R2016b (probably R2015 would also work). Additionally, only the makefile approach worked for me, not the cmake one. You also need to deactivate the cuDNN option from your Makefile.config file since the current version of cuDNN is incompatible with the "Caffe for Faster R-CNN" branch.

Please also note that the CPU mode can only be used for testing. I tried training on the CPU and the time required is literally prohibitive.

Best of luck!

[chenxiaoyu523]

non-existent field 'bottom_id_vecs' means that there is no definition in your code. bottom_idvecs is defined in older version caffe in caffe.cpp, and you need to recompile your mex-file.