lf1-io / padl

Functional deep learning
Apache License 2.0
106 stars 4 forks source link

Add a docstring/description property to the pipelines #447

Open eramdiaz opened 2 years ago

eramdiaz commented 2 years ago

🛰️ Feature

Description

One of the key points of PADL is the integration of the preprocessing and postprocessing into the dl pipeline, so we don't need to track these steps down in the original model and ensure we apply them when inferring.

However, sometimes we still need to inspect our pipeline either because it's been shared by a colleague or it's been some time since we worked with it to check the type of input that it's expecting, the steps that it's making or the meaning of the output. So, in order to enhance and complement the previous property of the PADL pipelines, it could be helpful to allow including a description of the pipeline that could specify the pipeline purpose, the type of input and the meaning of the output.

For example:

my_pipeline = some_prep >> batch >> layer >> unbatch >> some_post
my_pipeline.description = 
"""
this is a model for image classification which expects a 3x256x256 torch Tensor and outputs a 
dictionary with the three top labels and its score with the format

{
    'label_1': score_1,
    'label_2': score_2,
    'label_3': score_3 
}
"""
blythed commented 2 years ago

Good idea.

wuhu commented 2 years ago

:+1:, we should make sure not to violate our target of non-mutability. Perhaps we could extend the name operator (-) to accept more detailed descriptions.

my_pipeline = (
    some_prep >> batch >> layer >> unbatch >> some_post
) - """
this is a model for image classification which expects a 3x256x256 torch Tensor and outputs a 
dictionary with the three top labels and its score with the format

{
    'label_1': score_1,
    'label_2': score_2,
    'label_3': score_3 
}
"""