Having language modelling task as one of the task

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings

https://www.sbert.net

Apache License 2.0

15.23k stars 2.47k forks source link

Open thak123 opened 3 years ago

thak123 commented 3 years ago

Is it possible to have a language modeling task as one of the task in the multi-task setup when we train the sentence transformer ?

nreimers commented 3 years ago

Language modeling is currently not implemented as a loss. But when you create your own loss class, you can add it to the multi task setup

thak123 commented 3 years ago

Ahh... thanks for the lead... I will try to experiment in that direction