UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
14.83k stars 2.44k forks source link

Getting specific models, e.g. InferSent model #404

Closed githubrandomuser2017 closed 4 years ago

githubrandomuser2017 commented 4 years ago

I see on your documentation page that there are a variety of models we can use, e.g. roberta-large-nli-stsb-mean-tokens.

How are these model names related to the model files found under sentence_transformers/models/? I see files like BERT.py, T5.py, and LSTM.py.

I'm just looking to see if you have a pretrained model that uses the InferSent encoder (LSTM max pooling).

nreimers commented 4 years ago

The python files in sentence_transformers/models are building block to create sentence embedding models. For an InferSent Style model, you would need an LSTM layer followed by a Pooling layer with max_pooling.

For an example, see: https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/avg_word_embeddings/training_stsbenchmark_bilstm.py

These models need to be tuned on suitable data to produce meaningful sentence embeddings.

The pre-trained model page lists different pre-trained models. These models were trained on different datasets and produce meaningful sentence embeddings. They can be used without the need of any training.

Best Nils Reimers

githubrandomuser2017 commented 4 years ago

Thank you.