search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.38k stars 2.49k forks source link

evaluation #169

Open shainaraza opened 4 years ago

shainaraza commented 4 years ago

Thanks for sharing the great work and library, I have a confusion in my understanding of evaluation , please clarify for my understanding. 1)The evaluation metrics can only be used during pre-training? if I fine-tune a model for some task, lets say k-mean clustering, so how would I evaluate my results. 2) can I fine-tune on any dataset? thanks in advance

nreimers commented 4 years ago

Hi @shainaraza You would need to implement the evaluation yourself for your specific task.

For clustering for example, there is no standard way of evaluation. So you would need to find a way that fits your data & task.

Sure you can tune on any data set. But often only certain data sets & loss functions are useful. Fine-tuning is sadly still quite complex, as we target unsupervised tasks where we don't have labels and a loss we could propagate. So it is always trying around different things.

Best Nils Reimers

shainaraza commented 4 years ago

thanks you very much