search

wjun0830 / CGDETR

Official pytorch repository for CG-DETR "Correlation-guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding"
https://arxiv.org/abs/2311.08835
Other
115 stars 11 forks source link

Charades-STA performance replication #12

Closed keflanagan closed 3 months ago

keflanagan commented 3 months ago

HI, I've found that evaluating on the charades-sta dataset with the provided best charades checkpoint produces lower results than those reported in the paper. R@0.5 and R@0.7 are about 6% lower each. Do you have any idea why this might be happening? I've compared the individual predictions to the predictions provided in the google drive link and not all of the predictions are consistent with each other, which is presumably where the performance drop is coming from. Thanks.

wjun0830 commented 3 months ago

image Hello. Thanks for your interest in our work. When we tried with our checkpoint, it seemed to work fine.