Closed lambdaofgod closed 2 years ago
I get exactly the same error with sentence-transformers==2.1.0
when I run the very basic example from the doc:
from sentence_transformers import SentenceTransformer, InputExample
from torch.utils.data import DataLoader
model = SentenceTransformer('distilbert-base-nli-mean-tokens')
train_examples = [InputExample(texts=['My first sentence', 'My second sentence'], label=0.8),
InputExample(texts=['Another pair', 'Unrelated sentence'], label=0.3)]
train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=16)
next(iter(train_dataloader))
Error:
TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'sentence_transformers.readers.InputExample.InputExample'>
any news about this issue?
@milmin and @lambdaofgod I had a similar issue with the dataloader. The issue is that SentenceTransformer overwrites the default collator (https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/SentenceTransformer.py#L629) to work with the InputExample
class. So, that is why you are getting the error. If you want to test that your data pipeline is working you will need to import the smart_batching_collate
function and set it as your Dataloader's collate_fn
@ncoop57 I think it solves this problem, thanks.
In order to solve this
RuntimeError: 'lengths' argument should be a 1D CPU int64 tensor, but got 1D cuda:0 Long tensor
In addition to that we need to change LSTM class, like this issue suggests
smart_batching_collate
Thanks! That helped!
working code:
train_dataset,
shuffle=True,
batch_size=train_batch_size,
collate_fn=model.smart_batching_collate
)
smart_batching_collate
https://www.kaggle.com/code/rahulseetharaman/siamese-bert/notebook?scriptVersionId=102434046
I'm getting an error when I try to run training_stsbenchmark_bilstm.py example
I try to understand what's in the dataloader:
next(iter(train_dataloader))
I getWhole message:
Running the whole script
I get the following
Environment:
sentence-transformers==2.1.0 torch==1.8.0+cu111