deepset-ai / haystack

AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
17.72k stars 1.92k forks source link

YAML deserialization error in `OutputAdapter` when using `List` #8192

Closed oryx1729 closed 3 months ago

oryx1729 commented 3 months ago

Describe the bug YAML deserialization fails for OutputAdapter when using List in the output_type parameter. Here's an example:

from haystack import Pipeline

pipeline_yaml = """
components:
  output_adapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      output_type: "List[str]"
      template: "{{ test }}"
"""

Pipeline.loads(pipeline_yaml)

Error message

Traceback (most recent call last):
  File "../haystack/haystack/core/pipeline/base.py", line 186, in from_dict
    instance = component_from_dict(component_class, component_data, name, callbacks)
  File "../haystack/haystack/core/serialization.py", line 118, in component_from_dict
    return do_from_dict()
  File "../haystack/haystack/core/serialization.py", line 113, in do_from_dict
    return cls.from_dict(data)
  File "../haystack/haystack/components/converters/output_adapter.py", line 141, in from_dict
    init_params["output_type"] = deserialize_type(init_params["output_type"])
  File "../haystack/haystack/utils/type_serialization.py", line 117, in deserialize_type
    main_type = deserialize_type(main_type_str)
  File "/Users/tanaysoni/haystack/haystack/utils/type_serialization.py", line 142, in deserialize_type
    raise DeserializationError(f"Could not locate the type: {type_name} in the module: {module_name}")
haystack.core.errors.DeserializationError: Could not locate the type: List in the module: builtins

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../haystack/script.py", line 22, in <module>
    Pipeline.loads(pipeline_yaml)
  File "../haystack/haystack/core/pipeline/base.py", line 258, in loads
    return cls.from_dict(deserialized_data, callbacks)
  File "../haystack/haystack/core/pipeline/base.py", line 195, in from_dict
    raise DeserializationError(msg) from e
haystack.core.errors.DeserializationError: Couldn't deserialize component 'output_adapter' of class 'OutputAdapter' with the following data: {'type': 'haystack.components.converters.output_adapter.OutputAdapter', 'init_parameters': {'output_type': 'List[str]', 'template': '{{ test }}'}}. Possible reasons include malformed serialized data, mismatch between the serialized component and the loaded one (due to a breaking change, see https://github.com/deepset-ai/haystack/releases), etc.

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

To Reproduce Steps to reproduce the behavior

FAQ Check

System:

vblagoje commented 3 months ago

@oryx1729 would you please try using typing.List[str] i.e. the full class name. That should work, see https://github.com/deepset-ai/haystack/pull/8196 for more details. Let us know 🙏

sjrl commented 3 months ago

Yeah here is an example that worked for me in the past @oryx1729

adapter = OutputAdapter.from_dict(
    {
        'type': 'haystack.components.converters.output_adapter.OutputAdapter',
        'init_parameters':
        {
            'template': "\n{% set ns = namespace(doc_string='\n\n') %}\n{% for document in documents %}\n    {% set ns.doc_string = ns.doc_string + '[' + loop.index|string + ']: ' + document.meta.name + '\n' %}\n{% endfor %}\n{% set answer = [replies[0] + ns.doc_string] %}\n{{ answer }}\n",
            'output_type': 'typing.List[str]',
            'custom_filters': {}
        }
     }
)
vblagoje commented 3 months ago

Ok thanks @sjrl - I added a unit test just to have this use case one covered @oryx1729

vblagoje commented 3 months ago

Added a unit test, closing.