facebookresearch / InferSent

InferSent sentence embeddings
Other
2.28k stars 471 forks source link

KeyError: '</s>' #153

Open pradeepdev-1995 opened 1 year ago

pradeepdev-1995 commented 1 year ago

Tried the below code

from models import InferSent
import torch
model_version = 1
MODEL_PATH = "/home/encoder/infersent1.pkl"
params_model = {'bsize': 64, 'word_emb_dim': 300, 'enc_lstm_dim': 2048,
                'pool_type': 'max', 'dpout_model': 0.0, 'version': model_version}
model = InferSent(params_model)
model.load_state_dict(torch.load(MODEL_PATH))

use_cuda = False
model = model.cuda() if use_cuda else model

W2V_PATH = '/home/GLOVE/glove.6B/glove.6B.300d.txt'
model.set_w2v_path(W2V_PATH)

model.build_vocab_k_words(K=100000)

# Load some sentences
sentences = ["I ate dinner.",
       "We had a three-course meal.",
       "Brad came to dinner with us.",
       "He loves fish tacos.",
       "In the end, we all felt like we ate too much.",
       "We all agreed; it was a magnificent evening."]
with open('samples.txt') as f:
    for line in f:
        sentences.append(line.strip())
print(len(sentences))

embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True)
print('nb sentences encoded : {0}'.format(len(embeddings)))

But it showed the error KeyError: '</s>'. Where i got wrong?