search

deepset-ai / haystack

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
14.48k stars 1.7k forks source link

Add custom filter support to ConditionalRouter component #7940

Open vblagoje opened 5 days ago

vblagoje commented 5 days ago

*Is your feature request related to a problem? Please describe.** Currently, the ConditionalRouter component doesn't support custom filters in the 'condition' and 'output' fields of a route. This limitation restricts users from applying complex logic or data transformations within these fields, forcing them to pre-process data before it reaches the router or to create additional components for simple transformations. This can lead to less efficient pipelines and reduced flexibility in routing logic.

Describe the solution you'd like We propose adding support for custom filters in the ConditionalRouter, similar to how they are implemented in the OutputAdapter component. Specifically:

  1. Add a 'custom_filters' parameter to the ConditionalRouter's init method, accepting a dictionary of custom filter functions.
  2. Allow these custom filters to be used in both the 'condition' and 'output' fields of each route.
  3. Implement the necessary logic to apply these filters during the evaluation of conditions and generation of outputs.

This would allow users to write more expressive and powerful routing logic directly within the ConditionalRouter, enhancing its capabilities and flexibility.

Describe alternatives you've considered

  1. Pre-processing data: Users could transform data before it reaches the ConditionalRouter, but this can be cumbersome and less efficient for complex pipelines.
  2. Additional components: Creating separate components for data transformation before routing, which can make pipelines more complex and harder to manage.

Additional context This feature would align the ConditionalRouter more closely with the OutputAdapter in terms of functionality, providing a consistent experience across Haystack components. It would be particularly useful for scenarios involving complex data transformations or when integrating external libraries for specific computations within the routing logic.

ChrisPappalardo commented 5 days ago

Hi @vblagoje and team Haystack. I just implemented a custom component inheriting from ConditionalRouter that does this in a recent project. I'll submit a PR.

ChrisPappalardo commented 3 days ago

PR is up: https://github.com/deepset-ai/haystack/pull/7957