Open sonic182 opened 2 days ago
Hello!
The optimum-neuron
maintainers have indeed created modeling & trainer code compatible with AWS Neuron instances here: https://github.com/huggingface/optimum-neuron/tree/0ea72855252a1fec1c72b2e4773e09e00d4c65e2/optimum/neuron
But I'm afraid that in order to train/finetune Sentence Transformer models with AWS Neuron, then they would also have to create code mirroring the Sentence Transformers Trainer and related files. That is likely too much effort.
It is weird, considering that the model can be just moved to the xla device when training by using xla interface instead of compilation
Then perhaps training does work with Sentence Transformers out of the box? I haven't tested it, but perhaps you can just move the model to the desired device and see if a normal training script works.
Teorically it you're using transformers, it is possible to train in aws neuron instances (trn1)
With optimun neuron should be possible https://huggingface.co/docs/optimum/main/en/index, https://huggingface.co/docs/optimum-neuron/index
Have someone tried it?
Can be implemented?