get confidence score for each class

[gurkirt]

I am trying to get confidence score of reach class.

I am adding following lines below this

for (int c = 0; c <= numclasses; ++c){ top_data[count * 7 + 7+c] = scores[c]; }

Where else it would require more chnages? Without disturbing the current functional of computing mean AP etc.

[weiliu89]

What is the purpose of adding this? Isn't this enough?

[gurkirt]

Thanks for reply.

I need softmax scores for each class associated with the each box. I am working on spatio-temporal action detection. Something like this. I need softmax score vector for each box which will be used as uniary potential in solving association across frames.

I changed <a href"https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/detection_output_layer.cu#L121">this to top_shape.push_back(7+numclass); // numclass is number of classes including

Another change in previous comment for (int c = 0; c <= numclasses; ++c){ top_data[count * (7+numclass) + 7+c] = scores[c]; }

Now, net.forward() in python return me detection with shape (1, 1, num_det, 7+numclass) // where num_det is number of detections greater than threshold or topk detections.

but values of scores I get does not match.

for each a box is produced with [index label conf, xmin ymin xmax ymax score_0 score_1 .......... score_c] if label is 5 then score_5 shoul be == conf. Thats what I am not getting.

may be my understanding of score vector here is wrong. Do I need to extract scores for each class from all_conf_scores

[gurkirt]

following changes seems to work. for (int c = 0; c < numclasses; ++c){ const vector& tempscores = conf_scores.find(c)->second; // added this line and id in next line top_data[count * mconst + 7+c] = tempscores[idx];;//all_conf_scores[c][idx];conf_scores[c][idx]; }

[jmarwane]

Hi @gurkirt ! I'm trying to output top 5 scores for each prediction box instead of only best score. Which is very close to what you ere trying to do. Do you have an idea how i could do that simply ?

I'm trying to read detection_output_layer.cpp but i still can't fully understand how the scores are stored in conf_scores (or maybe scores).

Thanks !

[gurkirt]

@jmarwane It is bit long answer. I will answer it over the weekend.

[gurkirt]

Hi @jmarwane, I modified a bit detection.ipynb to access the score for all the classes in python.

Load caffe model

f = open(labelmap_file, 'r') labelmap = caffe_pb2.LabelMap() text_format.Merge(str(f.read()), labelmap)

net = caffe.Net(deploy_net_file, ### defines the structure of the model model_weights, ### contains the trained weights caffe.TEST) ### use test mode (e.g., don't perform dropout)

#### input preprocessing: 'data' is the name of the input blob == net.inputs[0] transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape}) transformer.set_transpose('data', (2, 0, 1)) transformer.set_mean('data', np.array(mean_values)) #### mean pixel transformer.set_raw_scale('data', 255) # the reference model operates on #### images in [0,255] range instead of [0,1] transformer.set_channel_swap('data', (2,1,0)) # the reference model has ####channels in BGR order instead of RGB

Forward pass

image = caffe.io.load_image(input_img_file) transformed_image = transformer.preprocess('data', image) net.blobs['data'].data[...] = transformed_image

Get confindence scores

detections = net.blobs['detection_out'].data[0][0] confidences = net.blobs['mbox_conf_softmax'].data[0]

Final detections

# Put detection bounding box and scores in single array

number_detection = np.shape(detections)[0]; final_detections = np.zeros((number_detection,num_classes+7)) for i in range(number_detection): final_detections[i,:7] = detections[i,:] final_detections[i,7:] = confidences[int(detections[i,0]),:]

## There are copies of some boxes being shown of different class ## So you have replicated detection boxes. ## To get the boxes ids change L155 of https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/detection_output_layer.cu#L155 to top_data[count * 7] = idx; ## And recompile caffe and pycaffe. Only do ### this when you need this indexes otherwise it will break training and ### validation process.

val,inds = np.unique(final_detections[:,0], return_index=True) final_detections = final_detections[inds,:];

Hope that helps

[seeyourcell]

@gurkirt “final_detections = np.zeros((number_detection,num_classes+7)) ” . Why 7? I think it mybe 6（top_conf,top_labels,top_xmin,top_ymin ,top_xmax ,top_ymax）.

[gurkirt]

It is 7, First entry is id of the prediction (or anchor box)（boxid, top_conf,top_labels,top_xmin,top_ymin ,top_xmax ,top_ymax）.

[weiliu89]

No, first one is image_id, not boxid

[gurkirt]

Yes in normal setting.

At test time you if you pass only one image and you can make following changes to get box ID. Like explained in above description.

There are copies of some boxes being shown of different class

So you have replicated detection boxes.

To get the boxes ids change L155 of

https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/detection_output_layer.cu#L155 to top_data[count * 7] = idx; ## And recompile caffe and pycaffe. Only do

this when you need this indexes otherwise it will break validation process at training time.

Every time you want to switch from testing to training you will need to reverse these changes and recompile.