Requesting checkpoints for cross-encoder/ms-marco-* for Bert-base or Bert-large models

[cramraj8]

Hi @nreimers

I would like to use Bert-base and Bert-large versions cross-encoder trained on ms-marco. I tried to fine-tune "cross-encoder/ms-marco-MiniLM-L-12-v2" on NQ and other standard datasets. But the fine-tuned performance drops. I suspect it's because of the actual objective of MiniLM is the distillation regression loss thus a cross-entropy based fine-tuning doesn't help with improved performance. Therefore, I am planning on fine-tuning the source bert-base/ bert-large/ albert-large that was initially used to distill MiniLM. Do you have those checkpoints ? And if so would you mind sharing them ? Thanks in advance!

[tomaarsen]

Hello!

I don't have access to the checkpoints, but I do want to point you to these places:

• https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco
• https://sbert.net/docs/pretrained-models/msmarco-v5.html

Perhaps there's some interesting information there for you.

• Tom Aarsen