Asymmetric semantic search question

[PrithivirajDamodaran]

I have a dataset of <context, query, score>, 30K triples. From the documentation, I understand that this comes under the rubric of Asymmetric semantic search with the context being a short passage.

As recommended I am planning to use MSMARCO trained with cosine sim as a base model.

Is it advisable to fine-tune using a Cross Encoder & CECorrelationEvalutor? The reason I am asking is I am wondering if adding a sequence classification head is better or just use the triples <context, query, score> in conjunction with say cosinesimilarityloss and play with the embedding space ?

Please advice

[nreimers]

Training the cross encoder will not provide a benefit for the embedding model.

What you could do is to train the cross encoder and then use this to add more training data or use MarginMSE Loss to train the bi-encoder.