search

jayleicn / singularity

[ACL 2023] Official PyTorch code for Singularity model in "Revealing Single Frame Bias for Video-and-Language Learning"
https://arxiv.org/abs/2206.03428
MIT License
129 stars 13 forks source link

About "Rerank dual-encoder results with cross-encoder“ #14

Closed ruiyan1995 closed 2 years ago

ruiyan1995 commented 2 years ago

https://github.com/jayleicn/singularity/blob/ef0cdbbcfa38ed9218565453d95e4dca25e2a5cb/tasks/retrieval_utils.py#L119

Hi, authors. Thanks for your great work. Could you provide any explanation or reference for this 're-ranking' trick? It seems to bring significant improvements on ret task, as shown below. 117821657473558_ pic .

FangmingZhou commented 2 years ago

The dual tower part does ranking first, after that, only the topK highest candidates can be reranked by the single-tower part. You can regard the single tower as a more concise but time-consuming module. So, it's not a trick, but two module explained in the paper

ruiyan1995 commented 2 years ago

@FangmingZhou Thanks for your reply very much! Could you point out the explanation in the paper? Sorry, I cannot find it.😢

FangmingZhou commented 2 years ago

@FangmingZhou Thanks for your reply very much! Could you point out the explanation in the paper? Sorry, I cannot find it.😢

The Method part in page 3: F_l(S) F_v(f_t) is the dual tower, H(*) is the single tower

FangmingZhou commented 2 years ago

You can learn more about this strategy from ALBEL paper