YoungXIAO13
commented
3 years ago Hi @lizhiyuanUSTC
Thanks for your question!
For the training procedure as well as the loss function, I simply follow the implementation proposed in Meta R-CNN (see this line).
It's true that the GroundTruth remains the same regardless of the class vector in training, while the output with highest classification score is selected in testing.
My understanding is that the box regression branch should be class-agnostic (as discussed in TFA), which means the regression output depends mainly on the RoI features with weak (or even zero) dependence on class feature. As for the classification branch, it tries to classify which class the object in RoI belongs to. This classification branch could work by only looking at the RoI feature and learning a specific weight for each class in the supervised setting. With the class feature vector, we can have four possible cases:
- the class vector and the output class is the object class, loss function requires a high classification score
- the class vector is object class while output class is not, loss function requires a low classification score
- the class vector is not object class while output class is, loss function requires a high classification score
- neither the class vector nor the output class is the object class, loss function requires a low classification score
By using the same class vector during training, we enforce the classification branch to be robust in all the cases without a specific relying on any configuration. This should work as the network could simply ignore the class feature vector and condition the output only on the RoI features when there are many samples, while adding class vector feature could surly help in few-shot cases (as demonstrated in FSRW and Meta R-CNN). In testing, as a class-specific prediction is required, for each combination of RoI and class, we can simply combine the RoI feature with the corresponding class vector feature.
lizhiyuanUSTC
Thanks for your great work!
I have some questions about the training and test setting. When training, no matter which class attention vector combined with the query feature, the groundtruth are same. However, the class score are connected with the attention vector when testing the performance and the score for background is from the first attention vector. The loss function did not build the connection between the predict class and the class of attention vector.
Is this strange?