Issue about the class's attentions of every roi of image in faster_rcnn.py.

[ztyxd]

In line 122 of the faster_rcnn.py, "proposal_labels = rois_label[b 128:(b + 1) 128].data.cpu().numpy()[0]" The proposal_labels here only take the label of the first RoI, not all the labels of the RoI.

Then in line 123 "unique_labels = list(np.unique(proposal_labels))"

It does not seek unique labels for all RoIs of the input image, but only for the first RoI. Then all RoIs are operated channel-wise multiplication with the attention of the specific class, but the class picked out in line 123 is not for all RoIs. I think this may be wrong, and it makes me very confused.

Hope you can give me some suggestions, thank you very much.

[YoungXIAO13]

Hi @ztyxd

Actually there are only 128 positive rois allocated for each input image, as configured here. And the variable b represent the number of images in each mini-batch, so it took all rois of an input image, not the first 128 RoIs :)

[ztyxd]

Hi YoungXIAO13

Thank you very much for your answer.

  My problem is that in the meta training stage, for each image in batch_size, it is necessary to filter the attention vector of the specific category according to the gt label to the RoI. (The relevant code is in line 114-171 in faster_rcnn.py)

 However, in line 122 in faster_rcnn.py（the code is shown as follows）

            proposal_labels = rois_label[b * 128: b + 1) * [128].data.cpu().numpy()[0]

It seems that it only took the first RoI label, not all the RoI labels

Then in line 123, the code is as follows

            unique_labels = list(np.unique(proposal_labels)) 

It does not seek unique labels for all RoIs of the input image, but only for the first RoI, because the "proposal_labels" now is just the label of the first RoI.

After the above two steps, all RoIs need to be multiplied with the selected attention vector. However, this selected attention vector is only based on the first RoI (line 122-123 in  faster_rcnn.py) not for all RoIs.

I think this maybe wrong, which makes me very confused.
Hope you can give me some suggestions, thank you very much

[YoungXIAO13]

I see, the [0] is sth I took directly from the code in Meta R-CNN, which might be indeed a bug. Maybe you can remove this and see would that change unique_labels ?

[ztyxd]

@YoungXIAO13 Well, thanks for your reply.

[pengyinxw]

Hi @ztyxd

Actually there are only 128 positive rois allocated for each input image, as configured here. And the variable b represent the number of images in each mini-batch, so it took all rois of an input image, not the first 128 RoIs :)

Hi @YoungXIAO13, as you said, so the 128 roi features are all positives, no backgrounds?

[ztyxd]

@pengyinxw You are right, the model selected all 128 RoIs corresponding to each input image, but the code only uses the first RoI when selecting unique_label. proposal_labels = rois_label[b 128: b + 1) [128].data.cpu().numpy()[0] I think the error caused by the "[0]".

For the 128 RoIs, I think there may contains a large number of negative samples.