search

leolee99 / PAU

The official implementation of paper "Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval" accepted by NeurIPS' 2023.
https://arxiv.org/abs/2309.17093
MIT License
21 stars 0 forks source link

Is the Re-ranking operation in eq.(7) used in the training stage? #3

Closed cty8998 closed 2 months ago

cty8998 commented 7 months ago

Is the Re-ranking operation in eq.(7) used in the training stage? If not, how to train the learnable parameters β1 and β2 in eq.(7)?

Besides, the video_sentence_logits and sentence_frame_logits are both used in the training and testing stage, is it right?

leolee99 commented 7 months ago

1) Thank you for your interest in our work. Re-ranking operation in eq. (7) is not used during the training stage. After training, $\beta_1$ and $\beta_2$ can be individually learned using simple optimization algorithms, such as Nelder-Mead and BFGS . We have integrated and updated the beta learning code into the repository. You can refer it at "main_pau.py: L432-L449"

2) Yes, both video_sentence_logits and sentence_frame_logits are used during the training and testing stages.