Difference between SBERT and DPR?

[jordane95]

Hello,

I recently read your BEIR paper and your SBERT lib doc, as well as the DPR paper. I find that DPR and SBERT share nearly the same architecture, except that SBERT tie the two encoders while DPR not. They also have the same training objective when performing dense retrieval task.

So is there any other difference between the two?

[nreimers]

They are quite similar:

• SBERT uses in most cases tied encoders, as this gives a better performance. DPR uses two independent encoders.
• SBERT mainly uses mean pooling, as it is better than the used CLS pooling in DPR
• DPR is fixed to bert-base, sentence-transformers can work with all encoder models.
• DPR used dot-product. In SBERT, there are models with cosine similarity as well as with dot product. Both measures have their pros and cons, so no way to decide which is better.
• The sentence-transformers framework comes with a lot more flexibility how to design your models and how to train them. So you can create a DPR like model also with SBERT.

[jordane95]

Thank you for your help !