search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.46k stars 2.5k forks source link

RuntimeError: FlashAttention is not installed #2981

Open seregadgl20-oss opened 1 month ago

seregadgl20-oss commented 1 month ago

Hi, can you tell me how to disable flash_attn? model = SentenceTransformer("jinaai/jina-embeddings-v3", device = device, trust_remote_code=True, model_kwargs={'default_task': 'text-matching' }) ................ trainer = SentenceTransformerTrainer( model=model, args=args, train_dataset=train_dataset, eval_dataset=eval_dataset, loss=train_loss, evaluator=dev_evaluator,

)

trainer.train() RuntimeError: FlashAttention is not installed. To proceed with training, please install FlashAttention. For inference, you have two options: either install FlashAttention or disable it by setting use_flash_attn=False when loading the model. Sentence Transformers v3.2

tomaarsen commented 1 month ago

Hello!

As also reported here: https://huggingface.co/jinaai/jina-embeddings-v3/discussions/47, I believe Flash Attention is required for training jina-embeddings-v3.