search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.46k stars 2.5k forks source link

crossencoder training on multigpu #2991

Open riyajatar37003 opened 1 month ago

riyajatar37003 commented 1 month ago

Hi,

I can see how to train embedding model with brand new trainer. but i couldn't see same for crossencoder. can you point me if i am missing ?

Thanks https://sbert.net/docs/sentence_transformer/training_overview.html

riyajatar37003 commented 1 month ago

https://sbert.net/docs/package_reference/cross_encoder/cross_encoder.html#training-inputs in this code , how can i save checkpoints after k-steps of training ?

tomaarsen commented 1 month ago

The CrossEncoder training hasn't yet been updated to use a new modern Trainer, that's still on my TODO-list, I'm afraid. There's currently not a very convenient way to save checkpoints every k steps. The easiest is actually to subclass the CrossEncoder, copy the fit method, and add a line like if training_steps % 1000 == 0: self.save(output_path + str(training_steps)) or something like it.

Apologies. I'll improve CrossEncoder support in the coming months.