search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
14.87k stars 2.44k forks source link

Plans for CrossEncoder and ColBERT #2912

Open ir2718 opened 3 weeks ago

ir2718 commented 3 weeks ago

Hi,

As the title says I'm interested in what are the planned improvements for the CrossEncoder and ColBERT in v3?

tomaarsen commented 3 weeks ago

Hello!

Good question. The v3 update introduced a Trainer for SentenceTransformer models, but left the CrossEncoder training the same. I plan to update the CrossEncoder training so that it also supports multi-GPU, bf16, loss tracking, etc.

Beyond that, I would like to add ColBERT support. I'm still not sure whether I should add that as a feature within the existing SentenceTransformer class, or whether I should separate it into a new ColBERT (or LateInteraction) class. A separation makes sense from a maintenance and "ease of use" perspective, but there might be a lot of value in joint Bi-Encoder & ColBERT models (a bit like BGE-M3) which can produce both sentence-level and token-level embeddings with just one inference.

emilysilcock commented 3 weeks ago

Hi @tomaarsen I was wondering if you have a sense of the timeline on this - especially the multi-GPU support for CrossEncoder. Thanks!

tomaarsen commented 2 weeks ago

Not at this time, I'm afraid.

ir2718 commented 1 week ago

@tomaarsen

Once you have more details please do share as I would love to contribute, especially with ColBERT.