search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.26k stars 2.47k forks source link

How to get sentence embeddings #68

Closed aszhanghuali closed 4 years ago

aszhanghuali commented 4 years ago

Hi!I just ask how to get sentence embeddings ?I have run training_nli_bert.py, but did not generate embeddings.

nreimers commented 4 years ago

Hi @aszhanghuali The training_nli_bert.py script just trains a new model. You can then use this model to generate sentence embeddings.

Training a model by yourself is not required, you can use one of the many available pre-trained models.

See this script how to generate sentence embeddings: https://github.com/UKPLab/sentence-transformers/blob/master/examples/basic_embedding.py

Best Nils

puttapraneeth commented 4 years ago

Hello,

Tried basic_embedding.py but it failed to find the model 'bert-base-nli-mean-tokens'. Details of the error are below. Could you please help in resolving this

FileNotFoundError Traceback (most recent call last)

in 1 from sentence_transformers import SentenceTransformer ----> 2 model = SentenceTransformer('bert-base-nli-mean-tokens') ~\Anaconda3\lib\site-packages\sentence_transformers\SentenceTransformer.py in __init__(self, model_name_or_path, modules, device) 74 logging.warning("You try to use a model that was created with version {}, however, your version is {}. This might cause unexpected behavior or errors. In that case, try to update to the latest version.\n\n\n".format(config['__version__'], __version__)) 75 ---> 76 with open(os.path.join(model_path, 'modules.json')) as fIn: 77 contained_modules = json.load(fIn) 78 FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Praneeth/.cache\\torch\\sentence_transformers\\public.ukp.informatik.tu-darmstadt.de_reimers_sentence-transformers_v0.2_bert-base-nli-mean-tokens.zip\\modules.json'
nreimers commented 4 years ago

You can find the models here: https://public.ukp.informatik.tu-darmstadt.de/reimers/sentence-transformers/v0.2/

Download them and unzip. Then change: model = SentenceTransformer('path/to/unzipped/model-folder')

The code is not that well tested for Windows, as a lot of frameworks have sadly only limited support for Windows.

Best Nils Reimers

puttapraneeth commented 4 years ago

Thanks for your swift response :)