search

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings
https://www.sbert.net
Apache License 2.0
15.24k stars 2.47k forks source link

T5 model #410

Open db1981 opened 4 years ago

db1981 commented 4 years ago

Hi, I tried to use the T5 model, as it is listed in the models package, but I get the following error:

File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 187, in encode out_features = self.forward(features) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/modules/container.py", line 100, in forward input = module(input) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(input, kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sentence_transformers/models/Transformer.py", line 25, in forward output_states = self.auto_model(features) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(input, *kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/transformers/modeling_t5.py", line 965, in forward decoder_outputs = self.decoder( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(input, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/transformers/modeling_t5.py", line 684, in forward raise ValueError("You have to specify either decoder_input_ids or decoder_inputs_embeds") ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

What am I missing?

Thanks!

nreimers commented 4 years ago

T5 does not work with AutoModel from hugginface transformers, as far as I know.

You could try to build your model from scratch and to use models.T5 instead of models.Transformers: https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/models/T5.py

In my experiments, T5 did not yield good sentence representations.

db1981 commented 4 years ago

T5 does not work with AutoModel from hugginface transformers, as far as I know.

Thank you!