deepset-ai / canals

A component orchestration engine
https://deepset-ai.github.io/canals/
Apache License 2.0
27 stars 3 forks source link

feat: add `Pipeline.inputs()` #120

Closed ZanSara closed 1 year ago

ZanSara commented 1 year ago

Related issues

https://github.com/deepset-ai/canals/issues/117

What

Example:

For this RAG pipeline, the function returns this dictionary:

    {
        'answer_builder': {
            'pattern': {
                'is_optional': True,
                'type': typing.Optional[str]
            },
            'query': {
                'is_optional': False, 
                'type': <class 'str'>
            },
            'reference_pattern': {
                'is_optional': True,
                'type': typing.Optional[str]
                }
        },
        'prompt_builder': {
            'question': {
                'is_optional': False, 
                'type': typing.Any
            }
        },
        'retriever': {
            'filters': {
                'is_optional': True,
                'type': typing.Optional[typing.Dict[str, typing.Any]]
            },
            'query': {
                'is_optional': False, 
                'type': <class 'str'>
            },
            'scale_score': {
                'is_optional': True,
                'type': typing.Optional[bool]
            },
            'top_k': {
                'is_optional': True, 
                'type': typing.Optional[int]
            }
        }
    }

and in error messages it prints the following

This pipeline accepts the following inputs:
- retriever:
    - query: str
    - filters: Optional[Dict[str, Any]]
    - top_k: Optional[int]
    - scale_score: Optional[bool]
- prompt_builder:
    - question: Any
- answer_builder:
    - query: str
    - pattern: Optional[str]
    - reference_pattern: Optional[str]
masci commented 1 year ago

I think this should be an utility function, not a method of the class. Same goes for draw FWIW.

ZanSara commented 1 year ago

@masci I like having them as methods for a bit of added discoverability (code completion basically) and on top of it, this syntax is identical to Haystack 1.x which can help the users. But I have no problem moving it out. If you confirm I'll move it out.

ZanSara commented 1 year ago

I slightly changed the style of this PR. I made two methods, one that returns the machine-readable version and one that prints to stdout. Methods implementations was moved into the validation.py file, because I'm now using the string representation to add information to some error messages as well.

Happy to remove/refactor everything as we see fit, let me know :slightly_smiling_face:


Example of the error message raised by a pipeline that received wrong input:

ValueError: Missing input: comp1.value

This pipeline expects the following inputs:
- comp1:
    - value: int
    - add: Optional[int]