search

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module
BSD 3-Clause "New" or "Revised" License
40 stars 27 forks source link

Sentence T5 Base Support #40

Closed trengrj closed 1 year ago

trengrj commented 1 year ago

When using sentence-t5-base as a custom model I get the following error:

FROM semitechnologies/transformers-inference:custom
RUN MODEL_NAME=sentence-transformers/sentence-t5-base ./download.py
curl -XPOST -H 'Content-Type: application/json' http://localhost:8084/vectors/ -d'{"text": "test"}'
{"error":"You have to specify either decoder_inputs or decoder_inputs_embeds"}

Traceback

t2v-transformers_1  | ERROR:    Something went wrong while vectorizing data.
t2v-transformers_1  | Traceback (most recent call last):
t2v-transformers_1  |   File "/app/./app.py", line 50, in read_item
t2v-transformers_1  |     vector = await vec.vectorize(item.text, item.config)
t2v-transformers_1  |   File "/app/./vectorizer.py", line 62, in vectorize
t2v-transformers_1  |     batch_results = self.model(**tokens)
t2v-transformers_1  |   File "/usr/local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
t2v-transformers_1  |     result = self.forward(*input, **kwargs)
t2v-transformers_1  |   File "/usr/local/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 1379, in forward
t2v-transformers_1  |     decoder_outputs = self.decoder(
t2v-transformers_1  |   File "/usr/local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
t2v-transformers_1  |     result = self.forward(*input, **kwargs)
t2v-transformers_1  |   File "/usr/local/lib/python3.9/site-packages/transformers/models/t5/modeling_t5.py", line 887, in forward
t2v-transformers_1  |     raise ValueError(f"You have to specify either {err_msg_prefix}inputs or {err_msg_prefix}inputs_embeds")
t2v-transformers_1  | ValueError: You have to specify either decoder_inputs or decoder_inputs_embeds
t2v-transformers_1  | INFO:     172.19.0.1:60954 - "POST /vectors/ HTTP/1.1" 500 Internal Server Error
trengrj commented 1 year ago

@antas-marcin ok some good news. When I test locally I can get this working.

My thought on fixing this is that we need to update the docker image transformers-inference:custom tag to point to a newer release with updated pytorch libraries.