search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.23k stars 2.47k forks source link

Can't load distiluse-base-multilingual-cased-v2 model #2924

Open b5y opened 1 month ago

b5y commented 1 month ago

Hello,

I am not able to load sentence-transformers/distiluse-base-multilingual-cased-v2 model. Script example:

from sentence_transformers import SentenceTransformer

model_name = 'sentence-transformers/distiluse-base-multilingual-cased-v2'

model = SentenceTransformer(model_name)

Raising error:

RuntimeError: Failed to import transformers.models.distilbert.modeling_distilbert because of the following error (look up to see its traceback):
libcudart.so.11.0: cannot open shared object file: No such file or directory

The same error is raised for the sentence-transformers/distiluse-base-multilingual-cased-v1 model, but everything works fine if I try to load the sentence-transformers/msmarco-bert-base-dot-v5.

Transformers version: 4.44.2 Sentence Transformers version: 3.0.1 Python version: 3.11.7 Torch version: 2.4.0

Do you have any thoughts on how to fix this error? I tried downgrading transformers to 4.44.0, but it didn't work.

I appreciate any help you can provide.

CC: @tomaarsen, @muellerzr

b5y commented 1 month ago

It looks like it solved the problem by removing flash-attn. Can't find a time to figure out why this didn't work with flash-attn with torch 2.4 because it worked in another env with torch 2.3.