search

zhou745 / GauFuse_WSTAL

20 stars 3 forks source link

Question about the mismatch of the code and Eq(3) in the paper. #12

Open zaie opened 1 month ago

zaie commented 1 month ago

Hello author, your work inspired me a lot! Tons of thanks:) I find your code does not match equations in your paper. In the function 'prediction_fuse_module', you use score_inner instead of oic_score to calculate the weights. Is there any reason for that? Obviously, oic_score is a better indicator for the importance of a proposal compared with score_inner. I'm so confused.

zhou745 commented 1 month ago

Hi, I cheked the code, its in the fusion is in utils/utils.py prediction_fuse_module line 174-177. The function 'prediction_fuse_module' is called in line 292 utils/utils.py, where first input element should be the predicted score in vid_cls_proposal (it is q in our paper). Line 174-177 is the fusion equation. Feel free to let me known if you have questions.

zaie commented 1 month ago

image the weights are np.exp(weights[id_seg]/T) and weights = segs[:, 3] as defined in line 141. Indeed, the vid_cls_proposal is the input weight. However, as defined in line 348, vid_cls_proposal[:,2] is the oic_score, ie, inner_score - outer_score. And the weights in line 174-177 are segs[:, 3], ie, vid_cls_proposal[:,3], which is the inner_score only. image

In your paper, the q is defined as the confidence score, which should be vid_cls_proposal[:,2] instead of vid_cls_proposal[:,3]. image

If my understanding is correct, the weights should be obtained based on confidence score (vid_cls_proposal[:,2]). However, the code actually uses inner score (vid_cls_proposal[:,3]) to obtain confidence score as shown in line141 and 174-177. Is this a bug or is there any reason for that?

zhou745 commented 1 month ago

Hi, thx for pointing this out. There is really nothing special about this score. However, replacing vid_cls_proposal[:,3] with vid_cls_proposal[:,2] should have a similar result(cause they are quite close, the inner-outer hyperparameter needs some change to get optimal performance. )