search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.42k stars 2.5k forks source link

Plans for CrossEncoder and ColBERT #2912

Open ir2718 opened 3 months ago

ir2718 commented 3 months ago

Hi,

As the title says I'm interested in what are the planned improvements for the CrossEncoder and ColBERT in v3?

tomaarsen commented 2 months ago

Hello!

Good question. The v3 update introduced a Trainer for SentenceTransformer models, but left the CrossEncoder training the same. I plan to update the CrossEncoder training so that it also supports multi-GPU, bf16, loss tracking, etc.

Beyond that, I would like to add ColBERT support. I'm still not sure whether I should add that as a feature within the existing SentenceTransformer class, or whether I should separate it into a new ColBERT (or LateInteraction) class. A separation makes sense from a maintenance and "ease of use" perspective, but there might be a lot of value in joint Bi-Encoder & ColBERT models (a bit like BGE-M3) which can produce both sentence-level and token-level embeddings with just one inference.

emilysilcock commented 2 months ago

Hi @tomaarsen I was wondering if you have a sense of the timeline on this - especially the multi-GPU support for CrossEncoder. Thanks!

tomaarsen commented 2 months ago

Not at this time, I'm afraid.

ir2718 commented 2 months ago

@tomaarsen

Once you have more details please do share as I would love to contribute, especially with ColBERT.

abedkhooli commented 2 months ago

I just came across Pylate , built on top of Sentence Transformers, which seems to implement ColBERT

abdelkareemkobo commented 3 days ago

I tried sentence-transformer multi process on my windows server with 8 2080TI GPU and it's working very fast even though, I am using pytorch 2.5.1 which removed LUB_UV and NVCC backend so i have to just work with gloo backend on my machine which i can't use with plain pytorch and sentence-transformers do it well. I can't use Ragatouille and other libraries :) So it will be very nice to see Sentence-transformers support models like (Colbertv2(Jina))

tomaarsen commented 1 day ago

@abdelkareemkobo Have you considered PyLate? It's essentially a wrapper around Sentence Transformers, and if ST ever implements ColBERT models it'll look very similar to PyLate.

abdelkareemkobo commented 1 day ago

@tomaarsen I invest the last two days with it and it works very fine thanks