search

wjun0830 / QD-DETR

Official pytorch repository for "QD-DETR : Query-Dependent Video Representation for Moment Retrieval and Highlight Detection" (CVPR 2023 Paper)
https://arxiv.org/abs/2303.13874
Other
207 stars 16 forks source link

Regarding results in Tables 1 and 2 of paper #4

Closed nogaini closed 1 year ago

nogaini commented 1 year ago

Hello,

I have a question regarding your paper - are the scores reported in Tables 1 and 2 with or without pretraining on ASR captions?

Best, Noga

wjun0830 commented 1 year ago

Those scores are without pretraining. Results with pretraining on asr can be found here

nogaini commented 1 year ago

Thank you for the quick response! :) I had a look at the w/ PT scores in PapersWithCode, and it seems they are referenced from the Highlight Detection results table and moreover, they seem to be scores obtained by other models. For example, the QD-DETR only Video w/ PT score seems to be linked to the SL-Module method by Xu et al. (2021) - column PR. Similarly, the QD-DETR w/ PT score is linked to the sLSTM method by Zhang et al. (2016) - column BK. Is there a mistake, or am I reading this the wrong way?

wjun0830 commented 1 year ago

Sorry, but I do not understand your question. Besides, in the paper, there are no results with ASR-pretrained models for our method.

nogaini commented 1 year ago

Sorry for not being clear with my question. What I meant to ask is - if you go here, and hover over the score values for "w/ PT", it seems to refer to scores which are completely unrelated to the QD-DETR model. So I was wondering what you meant when you said that the results with ASR pre-training can be found in the link you provided?

wjun0830 commented 1 year ago

Oh, sorry for confusing you. In the paper, we didn't actually report the results after ASR-pretraining. But for someone who might be interested in the results after ASR-pretraining, we pretrained QD-DETR with ASR captions first following the instructions on README and then trained QVhighlights dataset to obtain the numbers in the link. Details for ASR-pretraining can be found in Moment-DETR and if you follow the instructions in README to pretrain first and then train on QVHighlights, you sure may be able to reproduce the numbers in the link.

nogaini commented 1 year ago

Ah, I see! Sorry, I guess I just got confused with the interface of the PapersWithCode website. Thank you for the quick clarification! :) Closing this issue now as it's been resolved.