Error in training model (TripletLoss )

[naserahmadi]

Hello, I am trying to train a model. I am using TripletLoss :

warmup_steps = math.ceil(len(train_data)*num_epochs/train_batch_size*0.1)

model.fit(train_objectives=[(train_dataloader, train_loss)],
          evaluator=evaluator,
          epochs=4,
          evaluation_steps=100,
          warmup_steps=warmup_steps,
          output_path='data/distil')`

Can you help me with this error:

`TypeError                                 Traceback (most recent call last)
<ipython-input-9-c2b499a14191> in <module>()
      6           evaluation_steps=100,
      7           warmup_steps=warmup_steps,
----> 8           output_path='data/distil')

c:\workspace\sentence-transformers\sentence_transformers\SentenceTransformer.py in fit(self, train_objectives, evaluator, epochs, steps_per_epoch, scheduler, warmup_steps, optimizer_class, optimizer_params, weight_decay, evaluation_steps, output_path, save_best_model, max_grad_norm, fp16, fp16_opt_level, local_rank)
    403 
    404                 if evaluation_steps > 0 and training_steps % evaluation_steps == 0:
--> 405                     self._eval_during_training(evaluator, output_path, save_best_model, epoch, training_steps)
    406                     for loss_model in loss_models:
    407                         loss_model.zero_grad()

c:\workspace\sentence-transformers\sentence_transformers\SentenceTransformer.py in _eval_during_training(self, evaluator, output_path, save_best_model, epoch, steps)
    426         """Runs evaluation during the training"""
    427         if evaluator is not None:
--> 428             score = evaluator(self, output_path=output_path, epoch=epoch, steps=steps)
    429             if score > self.best_score and save_best_model:
    430                 self.save(output_path)

c:\workspace\sentence-transformers\sentence_transformers\evaluation\EmbeddingSimilarityEvaluator.py in __call__(self, model, output_path, epoch, steps)
     76 
     77                 emb1, emb2 = [model(sent_features)['sentence_embedding'].to("cpu").numpy() for sent_features in features]
---> 78 
     79             labels.extend(label_ids.to("cpu").numpy())
     80             embeddings1.extend(emb1)

c:\workspace\sentence-transformers\sentence_transformers\evaluation\EmbeddingSimilarityEvaluator.py in <listcomp>(.0)
     76 
     77                 emb1, emb2 = [model(sent_features)['sentence_embedding'].to("cpu").numpy() for sent_features in features]
---> 78 
     79             labels.extend(label_ids.to("cpu").numpy())
     80             embeddings1.extend(emb1)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\container.py in forward(self, input)
     90     def forward(self, input):
     91         for module in self._modules.values():
---> 92             input = module(input)
     93         return input
     94 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

c:\workspace\sentence-transformers\sentence_transformers\models\DistilBERT.py in forward(self, features)
     32         """Returns token_embeddings, cls_token"""
     33         # DistilBERT does not use token_type_ids
---> 34         output_states = self.bert(**features)
     35         output_tokens = output_states[0]
     36 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

TypeError: forward() got an unexpected keyword argument 'token_embeddings'`

[nreimers]

Have you tried using the latest version of sentence-transformers and transformers? Then, the error should disappear.

[naserahmadi]

I have the latest version

[nreimers]

Do you have the latest version of transformers (2.8.0)?

And how do you construct your model?

[naserahmadi]

Yes, transformers is update too. This code i use to create model: word_embedding_model = models.DistilBERT('distilbert-base-uncased')

pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension(),
                               pooling_mode_mean_tokens=True,
                               pooling_mode_cls_token=False,
                               pooling_mode_max_tokens=False)

model = SentenceTransformer(modules=[word_embedding_model, pooling_model])`

and code for train data:

train_batch_size = 16
num_epochs = 4
reader = TripletReader(dataset_folder='sentence-transformers/examples/datasets/test/',s1_col_idx=0,s2_col_idx=1,s3_col_idx=2)
train_data = SentencesDataset(reader.get_examples('test-train.csv'), model)
train_dataloader = DataLoader(train_data, shuffle=True, batch_size=train_batch_size)
train_loss = losses.TripletLoss(model=model,distance_metric=TripletDistanceMetric.COSINE)

[nreimers]

Looks right. Does this example work for you? https://github.com/UKPLab/sentence-transformers/blob/master/examples/training_transformers/training_wikipedia_sections.py

[naserahmadi]

You were right. I uninstalled it and installed it again and the error is gone. Now there is a new error:

ValueError                                Traceback (most recent call last)
<ipython-input-9-c2b499a14191> in <module>()
      6           evaluation_steps=100,
      7           warmup_steps=warmup_steps,
----> 8           output_path='data/distil')

c:\workspace\sentence-transformers\sentence_transformers\SentenceTransformer.py in fit(self, train_objectives, evaluator, epochs, steps_per_epoch, scheduler, warmup_steps, optimizer_class, optimizer_params, weight_decay, evaluation_steps, output_path, save_best_model, max_grad_norm, fp16, fp16_opt_level, local_rank)
    403 
    404                 if evaluation_steps > 0 and training_steps % evaluation_steps == 0:
--> 405                     self._eval_during_training(evaluator, output_path, save_best_model, epoch, training_steps)
    406                     for loss_model in loss_models:
    407                         loss_model.zero_grad()

c:\workspace\sentence-transformers\sentence_transformers\SentenceTransformer.py in _eval_during_training(self, evaluator, output_path, save_best_model, epoch, steps)
    426         """Runs evaluation during the training"""
    427         if evaluator is not None:
--> 428             score = evaluator(self, output_path=output_path, epoch=epoch, steps=steps)
    429             if score > self.best_score and save_best_model:
    430                 self.save(output_path)

c:\workspace\sentence-transformers\sentence_transformers\evaluation\EmbeddingSimilarityEvaluator.py in __call__(self, model, output_path, epoch, steps)
     73             features, label_ids = batch_to_device(batch, self.device)
     74             with torch.no_grad():
---> 75                 emb1, emb2 = [model(sent_features)['sentence_embedding'].to("cpu").numpy() for sent_features in features]
     76 
     77             labels.extend(label_ids.to("cpu").numpy())
ValueError: too many values to unpack (expected 2)

[nreimers]

Appears that you are not inputting a sentence pair dataset? You need at least two input sentences in order that EmbeddingSimilarityEvaluator works

[naserahmadi]

I have three sentences in my train file: sentence1 sentence2 sentence3 1 should be close to 2 and far to 3

@nreimers Wonder to know if you have any idea what is the problem here