search

deepset-ai / haystack

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
14.66k stars 1.72k forks source link

Input converters for T5 transformer not supported #4010

Closed obaidtambo closed 7 months ago

obaidtambo commented 1 year ago

Describe the bug an exception to the input converter was given.

Error message Exception: Exception while running node 'Generator': "Seq2SeqGenerator doesn't have input converter registered for t5-large. Provide custom converter for t5-large in Seq2SeqGenerator initialization" Enable debug logging to see the data that was passed when the pipeline failed.

Expected behavior produce results

Additional context Working with Long-Form Question Answering colab tutorials, I changed the model to t5-large in

generator = Seq2SeqGenerator(model_name_or_path="vblagoje/bart_lfqa")

To Reproduce from haystack.nodes import Seq2SeqGenerator

generator = Seq2SeqGenerator(model_name_or_path="t5-large")

FAQ Check

System:

obaidtambo commented 1 year ago 
          Closing this issue because Haystack supports T5 models in `TransformersSummarizer` https://github.com/deepset-ai/haystack/blob/d157e41c1fac351a01fe9348ecb2d9f7b84e29e7/haystack/nodes/summarizer/transformers.py#L23

and also in PromptNode: https://github.com/deepset-ai/haystack/blob/d2bba4935b2ccfa7ef875815a4a1bf98afcedbc1/haystack/nodes/prompt/prompt_node.py#L237 and in Seq2SeqGenerator for generative question answering: https://github.com/deepset-ai/haystack/blob/d2bba4935b2ccfa7ef875815a4a1bf98afcedbc1/haystack/nodes/answer_generator/transformers.py#L356

Originally posted by @julian-risch in https://github.com/deepset-ai/haystack/issues/701#issuecomment-1399480638

bogdankostic commented 1 year ago

Hi @obaidtambo! Our recommended way to use t5 models is to use our newly introduced PromptNode. There, you don't need to create a dedicated InputConverter but can quite easily choose one of the pre-defined prompts or use your own custom prompt.

Sorry, t5-large won't work with PrompNode. Only flan-t5 models work with PromptNode. To use t5-large with Seq2SeqGenerator, you need to create an input converter, similar to this one we have for BartEli5, and pass it to the input_converter parameter when initializing the Seq2SeqGenerator.