Can different embedding models use the same pretrained projector ?

[Clover-Hill]

Hi there, this is a very innovative paper and I really appreciate your work here.

In Appendix E, you compared the performance of different embedding models. Did you use the same projector or have you repeated the pretraining + fine-tuning process for each of these embedding models ?

I assume the output embedding spaces of different embedding models are very different and can't use the same projector. Have you tested the performance of a projector trained with retriever A but evaluated with retriever B ?

I guess my goal here is to use your pretrained projector but using a much smaller embedding model for my own project. Can you shed some light on how to achieve this ?

Thanks in advance.

[Hannibal046]

Hi, thanks for your interest in our work! In Appendix E, we train various projectors from scratch. I am afraid we can't reuse projector trained with embedding model A to adapt to embedding model B, simply because they are not in the same representation space.

[Clover-Hill]

Thanks for your response.

I understand it would be inconvenient for you to open source the complete data and pipeline for paraphrase pretraining and context-aware instruction tuning since follow-up works are being conducted.

I wonder if you could share some pretrained weight of projectors trained using small embedding models. I've sent an e-mail to chengxin1998@stu.pku.edu.cn regarding this matter. Please respond to my mail if you wish to discuss privately.