Support Dash prop callback connection closer to function signature

anders-kiaer commented 2 years ago

Is your feature request related to a problem? Please describe. When creating complex Dash dashbords/callbacks, there can be a long list of input/state/output. We in some cases have seen things of the size of:

@app.callback(
    Output("someid", "someprop"),
    Output("someid", "someprop"),
    Output("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    Input("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop"),
    State("someid", "someprop")
)
def _some_call_back(
    arg1,
    arg2,
    arg3,    
    arg4,    
    arg5,    
    arg6,    
    arg7,    
    arg8,    
    arg9,    
    arg10,    
    arg11,
    arg12,    
    arg13,    
    arg14,    
    arg15,    
    arg16,    
):
    ...

😱 Both during development, but also code review/maintenance, there is mental overhead with respect to seeing which (id, prop) pair belongs to which named argument in the function. This has gotten a bit better with https://dash.plotly.com/flexible-callback-signatures, but the underlying problem with (id, prop) and named argument in the function being defined in two different lines remains.

Describe the solution you'd like Utilize the Python 3+ typing syntax to annotate the (id, prop) closer to the argument. By using that we at the same time can avoid explicitly stating Output as that is already explicit by being defined as the returned type(s) of the function. The only thing to differentiate between is Input/State which could be done by some extra annotation tag. E.g. something like

from typing import Annotated as A

@app.callback
def _some_call_back(
    arg1: A[List[str], "someid", "someprop"],
    arg2: A[List[str], "someid", "someprop"],
    arg3: A[List[str], "someid", "someprop"],
    arg4: A[List[str], "someid", "someprop"],
    arg5: A[List[str], "someid", "someprop", "state"],
) -> Tuple[
        A[List[str], "someid", "someprop"],
        A[List[str], "someid", "someprop"],
        A[List[str], "someid", "someprop"],
    ]:
    ...

Another benefit is that we encourage Dash app developers to type hint their callback functions which again helps IDEs and mypy detect bugs (related to #1786).

Additional context

See https://docs.python.org/3/library/typing.html#typing.Annotated for documentation on typing.Annotated (backported to earlier Python 3 versions in typing_extensions).

Related to #1748 and #1786, but implementation is independent of those.

PoC code showing how type annotations potentially can be transformed into the Input/Output/State when callbacks are registered/decorated:

The below code will print out ``` Output('id1', 'propname') Output('id2', 'propname') Input('id3', 'propname') Input('id4', 'propname') State('id5', 'propname') State('id6', 'propname') State('id7', 'propname') ``` ```python ############# # Dash core # ############# import inspect # In the std.lib typing library on Python 3.8+: from typing_inspect import get_args, get_origin def callback(func): for name, annotation in inspect.getfullargspec(func).annotations.items(): if name == "return": if get_origin(annotation) == tuple: # Multi valued output for return_value in get_args(annotation): print(f"Output{return_value.__metadata__}") else: print(f"Output{annotation.__metadata__}") else: if len(annotation.__metadata__) >= 3 and annotation.__metadata__[2] == "state": print(f"State{annotation.__metadata__[:2]}") else: print(f"Input{annotation.__metadata__}") return func ###################### # Dash app developer # ###################### from typing import List, Tuple from typing_extensions import Annotated as A # On Python 3.9+ this is in typing std.lib @callback def _update_plot( arg3: A[List[str], "id3", "propname"], arg4: A[List[str], "id4", "propname"], arg5: A[List[str], "id5", "propname"], arg6: A[List[str], "id6", "propname", "state"], arg7: A[List[str], "id7", "propname", "state"] ) -> Tuple[ A[List[str], "id1", "propname"], A[List[str], "id2", "propname"] ]: ... ```

emilhe commented 2 years ago

I really like this idea. I just put together a simple (but working) prototype implementation inspired by your code. So far, the syntax looks like this,

from dash_extensions.enrich import T, DashProxy, callback, html

@callback()
def hello(n_clicks: T[int, "btn", "n_clicks"]) -> T[str, "log", "children"]:
    return "Hello world!"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([html.Button("Click me", id="btn"), html.Div(id="log")])

if __name__ == '__main__':
    app.run_server()

where T is the import name for Annotated (where "T" is short for "Type", it could also just have been "A", or maybe "P" for "Prop"). My immediate idea was to create Ouput/Input/State classes by subclassing Annotated, but it seems this is not possible. What do you think, @anders-kiaer ? Do you have any ideas for improvements? :)

emilhe commented 2 years ago

Another option would be to use a syntax closer to the current one, e.g. something like

from dash_extensions.enrich import DashProxy, callback, html, Input, Output

@callback
def hello(n_clicks: Input("btn", "n_clicks")) -> Output("log", "children"):
    return "Hello world!"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([html.Button("Click me", id="btn"), html.Div(id="log")])

if __name__ == '__main__':
    app.run_server()

This approach doesn't allow the use of actual type annotations though, which might be a drawback. On the other hand, native Dash doesn't allow complex types as input/output for callbacks, so it might not be so bad after all.

anders-kiaer commented 2 years ago

I really like this idea. I just put together a simple (but working) prototype implementation inspired by your code.

🎉🙂

Do you have any ideas for improvements? :)

Overall I like the syntax and think using type hint annotations improves readability. I'm not sure if the following are ideas, maybe more questions:

(Input/State) and Output are clearly distinguished by being input and output type hints of the callback function. How do we best distinguish between Input and State within the input arguments?
from dash_extensions.enrich import T - did you encounter issue with e.g. from typing import Annotated as A from std.lib?
Reading the Python standard library documentation of typing.Annotated, they have examples of using typing.Annoted[sometype, SomeClass(initval)], maybe because it is easier to distinguish then with isinstance (as they also use as example) should the end user have other use cases for annotating the arguments, in addition to dash (id, prop) specification (in pydantic they have some discussion on e.g. argument text descriptions as annotations). E.g.:
```
from dash.dependencies import Prop

@callback
def hello(
  some_input: T[int, Prop("id", "prop")],
  some_state: T[int, Prop("id", "prop", trigger=False)],
) -> T[str, Prop("id", "prop")]:
  return "Hello world!"
```

On the other hand, native Dash doesn't allow complex types as input/output for callbacks, so it might not be so bad after all.

My hope is that it will change after #1786 - and then it would be nice to able to use the full power of type hint annotations. 🙂

One option (combining the class approach above and also reuse existing Input/State/Output) could be

@callback
def hello(n_clicks: A[sometype, Input("btn", "n_clicks")]) -> A[sometype, Output("log", "children")]:
    return "Hello world!"

but it feels more verbose/explicit than necessary (Output is already clearly an output by being the returned type of the function).

emilhe commented 2 years ago

I have done another (small) iteration of the syntax, partly based on your comments, @anders-kiaer . My working example now looks like this,

from dash_extensions.enrich import DashProxy, callback, html, dcc, prop

@callback
def hello(n_clicks: prop("btn", "n_clicks", int), # the type definition "int" is optional
          name: prop("name", "value", trigger=False)) -> prop("log", "children"):
    return f"Hello {name}! (click count is {n_clicks})"

app = DashProxy(prevent_initial_callbacks=True)
app.layout = html.Div([
    dcc.Input(placeholder="Enter your name here", id="name"),
    html.Button("Click me!", id="btn"),
    html.Div(id="log")
])

if __name__ == '__main__':
    app.run_server()

A few notes on the syntax,

The Input/Output/State objects are replaced by a common prop function
The first two arguments of the prop function are component id and component property (both mandatory), the third argument is an optional type definition (which just defaults to any)
All argument props are per default interpreted as Input, but can be changed to State by passing trigger=False
Similarly, all return props are interpreted as Output, but can be changes to ServersideOutput by passing serverside=True

You should be able to try out the syntax with the following rc release,

https://pypi.org/project/dash-extensions/0.0.67rc2/

I haven't tested much more than the example above, so expect dragons :)

AnnMarieW commented 2 years ago

Hey @anders-kiaer and @emilhe

I'd be interested to hear your feedback on #1952 Improved callback_context - especially the section "Bonus 1: Simplified callback function signatures" section

plotly / dash

Support Dash prop callback connection closer to function signature #1810