search

huggingface / setfit

Efficient few-shot learning with Sentence Transformers
https://hf.co/docs/setfit
Apache License 2.0
2.24k stars 222 forks source link

Compatibility with ColBERT Models Trained with Pylate #563

Closed abedkhooli closed 1 month ago

abedkhooli commented 1 month ago

Tried to use a ColBERT model trained using Pylate since it was built on top of Sentence Transformers. The model loaded (SetFitModel.from_pretrained) but failed in trainer.train(): [/usr/local/lib/python3.10/dist-packages/sentence_transformers/losses/CosineSimilarityLoss.py](https://localhost:8080/#) ---> 76 embeddings = [self.model(sentence_feature)["sentence_embedding"] for sentence_feature in sentence_features] KeyError: 'sentence_embedding' Are those models fully incompatible or there is a fix?

tomaarsen commented 1 month ago

I'm afraid those are fully incompatible. ColBERT models simply have different outputs than regular Sentence Transformer models.