Closed aoezdTchibo closed 1 week ago
Hey, you are unfortunately not able to use model2vec
like this. Please use the following snippet:
from sentence_transformers import SentenceTransformer
from sentence_transformers.models import StaticEmbedding
model_name = "intfloat/multilingual-e5-small"
static_embedding = StaticEmbedding.from_distillation(model_name, device="cuda")
model = SentenceTransformer(modules=[static_embedding])
Should you have any further questions, please don't hesitate to reach out.
Works with the provided snippet, thank you!
If I try to use the distilled model with Sentence Transformers, I am getting the following error:
Use the following code snippet to reproduce the error: