search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.13k stars 2.46k forks source link

Training/Finetune in trn1 #3010

Open sonic182 opened 2 days ago

sonic182 commented 2 days ago

Teorically it you're using transformers, it is possible to train in aws neuron instances (trn1)

With optimun neuron should be possible https://huggingface.co/docs/optimum/main/en/index, https://huggingface.co/docs/optimum-neuron/index

Have someone tried it?

Can be implemented?

tomaarsen commented 2 days ago

Hello!

The optimum-neuron maintainers have indeed created modeling & trainer code compatible with AWS Neuron instances here: https://github.com/huggingface/optimum-neuron/tree/0ea72855252a1fec1c72b2e4773e09e00d4c65e2/optimum/neuron

But I'm afraid that in order to train/finetune Sentence Transformer models with AWS Neuron, then they would also have to create code mirroring the Sentence Transformers Trainer and related files. That is likely too much effort.

sonic182 commented 2 days ago

It is weird, considering that the model can be just moved to the xla device when training by using xla interface instead of compilation

tomaarsen commented 2 days ago

Then perhaps training does work with Sentence Transformers out of the box? I haven't tested it, but perhaps you can just move the model to the desired device and see if a normal training script works.