How to reproduce results with Universal Sentence Encoder for STS tasks? Is the model available in the library? Unable to find it.

[rajat-tech-002]

Can you please provide the link to the Universal Sentence Encoder model that was used to calculate results for STS tasks? How to fine-tune USE model?

[nreimers]

See: https://tfhub.dev/google/universal-sentence-encoder-large/5

Code for fine-tuning is not available as far as I know

[rajat-tech-002]

See: https://tfhub.dev/google/universal-sentence-encoder-large/5

Code for fine-tuning is not available as far as I know

But @nreimers, so USE model was not fine-tuned for producing the results in the paper - ?

[nreimers]

The USE model was fine-tuned on certain data (more in the linked paper), like QA questions from various internet forums, NLI data etc.

But for these results, I used the provided pre-trained models as there is no training code available for USE.

[rajat-tech-002]

One more question @nreimers. Thanks for the reply. Is the model 'distiluse-base-multilingual-cased-v2' same as USE model? And can it be fine-tuned? It is given in the following link: https://www.sbert.net/docs/pretrained_models.html

[nreimers]

It used the approach here: https://arxiv.org/abs/2004.09813

As teacher, USE-multilingual was used, as student, distilbert-multilingual was used. Hence, it produces fairly similar embeddings as USE, but:

• It works for more languages (50+ languages instead of 16 languages as the original USE-multilingual)
• And yes, it can be fine-tuned.