Clientside Callbacks - Githubissues

chriddyp commented 6 years ago

In Dash, changes to component properties in the front-end trigger user-supplied Python functions (@app.callback). This framework is very flexible: users have full control over the Python code that they write in their app.callback functions.

However, app.callback functions are simple. They filter some data or change the color of the chart or display some text. These data transformations, although simple, currently need to be processed entirely over the network in the Python backend. This makes the apps slower than they could be (because of the network delay) and less portable than they could be (because they require a running Python server).

Clientside Callbacks will introduce an interface for describing data transformation relationships between components in JavaScript. These data transformations will happen entirely in the client-side without passing data over the network. The Clientside callbacks Dash framework will enable developers to swap out their python-driven @app.callback functions with javascript-executed data transformations, enabling more performant apps.

As a quick background, here is what Dash Apps currently look like in Python:

The first part describes what the app looks like. These classes just declaratively describe the names and the props of the React components that they generate. These objects get serialized as JSON.

app.layout = html.Div([

    html.H1('Example'),

    comonents.TextInput(
        id='my-text-input',
        value='Initial value'
    ),
    components.Dropdown(
        id='my-dropdown',
        options=[
            'Option A', 'Option B', 'Option C'
        ]
        value='Option A'
    ),

    components.Graph(
        id='my-graph'
        figure={             # Just a plotly figure
            'data': [...],
            'layout': {...}
        }
    )
])

The second part of dash apps describe the relationship between graphs. This sets up "sources" ("inputs") and "sinks" ("outputs") in the Dash front-end. Whenever any of the Input properties change, the AJAX call gets made.

It's Reactive like a spreadsheet: as inputs change, the new values propagate down the dependency graph, updating components in the correct order:

1_basic_reactive

@callback(
    Input(
        id='my-dropdown',
        property='value'
    ),
    Input(
        id='my-text-input',
        property='value'
    ),
    Output(
        id='my-graph',
        property='figure'
    )
)
def update_graph(new_dropdown_value, new_text_input_value):
    # compute a new figure based off of the new values of the dropdown
    # or text input
    return {
        'data': [{
            'x': [1, 2, 3],
            'y': ({
                'Option A': [3, 1, 2],
                'Option B': [4, 3, 5],
                'Option C': [1, 2, 4]
            })[dropdown]
        }],
        'layout': {'title': text}
    }

These reactive updates happen entirely server-side through HTTP requests. (This allows dash developers to do complicated updates or analytics through their python context).

I think that we could extend this framework to work client-side as well. Instead of a custom function defining how Inputs ("sources") update Outputs ("sinks"), we could define a library of transformations components and a syntax for relating input properties to output properties. These transformations could be just be functional React components.

Here are some conceptual examples: 2_input_updates_text

from dash.serverless import Selector as S

layout = Div([
    Input(id='my-input', value='initial-value'),
    H3(children=S('my-input', 'value'))
])

In this example, we're setting the "children" property of the HTML H3 element to just be the "value" of the "my-input" component. When "value" changes, the content of the H3 element updates to match the new value.

I'm wrapping the ID and property with S to denote that the string represents a "reactive" property corresponding to the component with the specified ID and that component's property. (The actual API might be different, just using s for conceptual purposes.)

Now, consider a "Dataset" component and a "Graph":

3_dataset_graph

layout = Div([
    Dataset(
        id='my-dataset'
        columns={
            'column-1': [1, 2, 3],
            'column-2': [3, 1, 4]
        },
        column_names={
            'column-1': 'My first column',
            'column-2': 'My second column'
        }
    ),

    Graph(
        figure={
            'data': [{
                'x': S('my-dataset', 'column-1'),
                'y': S('my-dataset', 'column-2')
            }]
        }
    )
])

Note that not all components actually get rendered in the DOM. In this case, the Dataset component isn't actually visible. It's just included as state. If you wanted to view it as a table, it would look like:

layout = Div([
    Dataset(
        id='my-dataset'
        columns={
            'column-1': [1, 2, 3],
            'column-2': [3, 1, 4]
        },
        column_names={
            'column-1': 'My first column',
            'column-2': 'My second column'
        }
    ),

    Table(data='::my-dataset.columns'),

    Graph(
        figure={
            'data': [{
                'x': S('my-dataset', 'columns', 'column-1'),
                'y': S('my-dataset', 'columns', 'column-2')
            }]
        }
    )
])

You can imagine how there might be several datasets and several graphs in one (like a dashboard or a report).

layout = Div([
    Dataset(id='dataset-1', columns={...}),
    Dataset(id='dataset-2', columns={...}),
    Dataset(id='dataset-3', columns={...}),
    Graph(id='graph-1',
          data=[{'x': S('dataset-1', 'columns', 'column-1'), ...}]
    ),
    Graph(id='graph-2',
          data=[{'x': S('dataset-2', 'columns', 'column-1'), ...}]
    ),
    Graph(id='graph-3',
          data=[{'x': S('dataset-3, 'columns', 'column-1'), ...}]
    )
])

Now, we would also need a library for lightweight data transformations. I'm thinking something like Ramda.

import dash.clientside.transformations as T
import dash.clientside.selector as S

df = pd.DataFrame([
    {'col-1': 1, 'col-2': 5, 'col-3': 10},
    {'col-1': 2, 'col-2': 6, 'col-3': 11},
    {'col-1': 3, 'col-2': 7, 'col-3': 12},
    # ...
])

app.layout = html.Div([

    # "Virtual" component that doesn't render anything
    # to the screen, it just contains the data for other
    # components to reference
    dcc.Dataset(
        id='my-dataset',
        columns=df.columns,
        rows=df.to_dict(),
    ),

    # `Table` renders an actual table to the screen
    dcc.Table(
        rows=S('my-dataset', 'rows'),
        columns=S('my-dataset', 'rows')
    ),

    dcc.Graph(
        figure={

            # T.pluck('col-1', [{'col-1': 1}, {'col-1': 2}]) -> [1, 2]

            'x': T.pluck(
                'col-1',
                S('my-dataset', 'rows')
            ),
            'y': T.pluck(
                'col-2',
                S('my-dataset', 'rows')
            )
        }
    )
])

Or, extending this with dynamic components:

app.layout = html.Div([
    dcc.Dataset(
        id='my-dataset',
        columns=df.columns,
        rows=df.to_dict(),
    ),

    dcc.Dropdown(
        id='my-dropdown',
        options=[
            {'option': i, 'label': i}
            for i in df.columns
        ],
        value=df.columns[0]
    ),

    dcc.Graph(
        figure={
            'x': T.pluck(
                'col-1',
                S('my-dataset', 'rows')
            ),
            'y': T.pluck(
                S('my-dropdown', 'value'),
                S('my-dataset', 'rows')
            )
        }
    )
])

5_simple_declarative_dropdown

Here are some high-level architectural requirements and goals for this work item:

The Dash Apps will still be created with Python
The clientside callbacks will be executed in JavaScript
The existing set of Dash components will be available (with the exception of network-connected components like the mapbox charts)
We will introduce a new syntax or language ("data transformation language") for declaratively describing the relationships and simple operations between component properties. This language will have a Python interface, will be serialized as JSON, and executed in JavaScript.
This client-side data transformation will be available in server-connected Dash apps as well, enabling certain simple updates to happen quickly
The app will not run arbitrary JavaScript, it will be designed in a way to be safe from XSS injections
The "data transformation language" will operations like filtering, plucking values from nested objects, sorting, and arithmetic. It will draw inspiration from functional programming libraries and languages that enable concise, chainable, and immutable data transformations. See Ramda (http://ramdajs.com/) for an example.

ned2 commented 6 years ago

This is super exciting, as it will greatly increase the range of contexts where Dash can be used. The plan to have serverless components be available within a server-connected app is a great idea. I've often found myself swapping bits of pre-computed layout in and out using callbacks. Having to do an entire round-trip just to change the page layout is not ideal. This will allow for much more responsive single-page apps.

valentijnnieman commented 6 years ago

In theory, this would open up Dash apps to be used with different React renderers, such as these https://github.com/chentsulin/awesome-react-renderer right? Very exciting!

Tasselmi commented 6 years ago

hey , guys. I think you make problems complicated.
Just look at plotly.offline.plot() . I think you should not create any new language at all . Just create a function or a package that can generate offline html files ,ok ? For example, we use dash as now and we view the app by visiting https://127.0.0.0:8000 or some address like this , a function u provide can download all the data and web UI and interactive features in the page and generate a offline html file . Studying a new language costs a lot time and the offline mode is quite different from now . I am afraid.

chriddyp commented 6 years ago

Thanks for chiming in @Tasselmi !

a function u provide can download all the data and web UI and interactive features in the page and generate a offline html file

This will be possible if you want to just export the app.layout of your app. Nothing new to learn here.

However, if you want to convert your Python callbacks to a standalone file, we need some way to run those callbacks without a Python server running. So, that's what this "new language" is proposing: a way to write your Dash callbacks in a way that can be run without a Python server.

davidolmo commented 6 years ago

I was looking for this feature before trying out Dash. This would be great

Jonnyb36 commented 6 years ago

This will be possible if you want to just export the app.layout of your app. Nothing new to learn here.

This would also be a really useful feature by itself as often I would just like to take a snapshot of the html with the existing widget states that have generated that html.

eruisi commented 6 years ago

This is a feature missing from Dash compared to Bokeh autload_static mode. Very excited to see it will come to Dash too!

Do you have an ETA for the feature?

chriddyp commented 6 years ago

Do you have an ETA for the feature?

Shooting for late fall, but I'll keep this post updated with progress.

Glad you're excited, it's going to be sweet :)

Tasselmi commented 6 years ago

Hey guys. How is all of this going? I think offline mode is historic. As a data analyst, i will never use Microsoft office any more.

chriddyp commented 6 years ago

I think offline mode is historic. As a data analyst, i will never use Microsoft office any more.

That's what I'm talking about!

It's looking good. I promise to keep this thread updated with developments. All I can say now is that we're shooting for late fall.

Tasselmi commented 6 years ago

Thank u for ur response.

Immortalin commented 6 years ago

@chriddyp Easy way: https://github.com/QQuick/Transcrypt

Throw in pyodide if you want the Scipy kitchen sink. Having a custom DSL is nice yes, but having the full power of Python is even nicer. :D

Immortalin commented 6 years ago

https://github.com/doconix/pyreact/blob/master/src/scripts/clock.py

Needs a bit of work to smooth out the rough edges of React but I think you guys should seriously consider this method.

mullenkamp commented 6 years ago

Hi, I just wanted to chime in to say that I would love to see this feature as well. Using a standalone dash/plotly plot will be great for relatively lightweight plots with callbacks as a server would likely be overkill. Thanks for all your great work!

viogp commented 5 years ago

While this feature is being develped, I'd like to simply access the htm image generated with dash. I've tried saving the page as htm, following the suggestion from @chriddyp that I saw in a related thread. However, when I load the file, this lasts less than a second on my browser and afterwards I get:

Error loading layout

I'm using firefox on a Windows 10 machine.

kknckk commented 5 years ago

@chriddyp how are things goin? winter is coming 😟

Tasselmi commented 5 years ago

@chriddyp how are things goin? winter is coming 😟

haha 😁 winter is coming~~ game of the th~~~

chriddyp commented 5 years ago

how are things goin?

Things are going well. I promise that I will keep this thread updated with details and progress, so please hang tight.

stippingerm commented 5 years ago

I would like to implement some custom interaction in an plotly.offline (or dash) figure. This means embedding custom JS code from python into the generated figure that is hooking directly to the underlying plotly.JS. @chriddyp whet is offered in this thread is exciting, however, to me it looks too complicated for the simple purpose. To support this:

embedding custom JS into the figure from python script is possible but awkward, see here.
this embedding should not be too difficult to support officially, but I have not found any doc on that (in addition, it is a standard feature in Bokeh, but now I need 3D scatter plot not available there)
to achieve my goal before Serverless Dash Apps is finished I think I would need only an explicit documentation how the data passed from python to html is exposed to the JS script (e.g., array name and scope, field names, data format, etc.)

Please, could you give me a proper reference how python code is translated (especially, how the user-provided data is exposed/available in JS) when using plotly.offline.

ccirone2 commented 5 years ago

Count me in as an someone looking forward to serverless Dash apps! Interactive offline plots are easy to share and offer so much utility. I would love to be able to go the step further to incorporate callback features in a standalone file that I can share in an email. Having to use a service to host a dash app is just too much overhead for me to overcome in my work ecosystem.

pgr-gallup commented 5 years ago

Are you still working on this? Would a gamechanger for me.

edwardreed81 commented 5 years ago

This would be huge for me as well. Is it still on the timeline?

betizad commented 5 years ago

This feature would a great addition to our project too. Looking forward to it coming.

chriddyp commented 5 years ago

https://github.com/plotly/dash/pull/672

betizad commented 5 years ago

672

Can I check out this branch and test it? I have not done so and only used package installtion via conda/pip. Is it enough to clone dash-renderer, switch to branch clientside-2, and install with "python .\setup.py install"?

Btibert3 commented 5 years ago

Any update on this?

chriddyp commented 5 years ago

Done in #672

mungojam commented 5 years ago

If anybody puts together a fully serverless dash app using this, it would be great to see an example. (I may try but can't guarantee that I'll get time for it)

vlizanae commented 5 years ago

is there any kind of documentation for this already?

davidolmo commented 5 years ago

I've seen the examples and it looks promising. If all callbacks are performed clientside, do you think there would be any way to export the Dash app to html+.js in order to allow offline usage?

louisdecharson commented 5 years ago

Documentation

For those of you looking for an example, one can be found here: https://community.plot.ly/t/dash-0-41-0-released/22131

mungojam commented 5 years ago

Documentation

For those of you looking for an example, one can be found here: https://community.plot.ly/t/dash-0-41-0-released/22131

I don't know about others, but I was asking for an example (if it's possible) of how to use this to generate a fully clientside app. I suspect it isn't possible at this stage, but would be good for the future as it enables things like s3 hosting of some basic dash apps. Though maybe we would gain very little in using dash over pure react at that point.

alexcjohnson commented 5 years ago

how to use this to generate a fully clientside app. I suspect it isn't possible at this stage

Right, for now this feature is focused on performance, and does not provide a pathway to a fully serverless architecture.

mneira10 commented 5 years ago

Does this let me get the corresponding html of a dash plot? I haven't been able to get the raw html yet and it is not clear to me how it can be done with this feature.

Has anyone been able to do this?

giiyms commented 5 years ago

Documentation

For those of you looking for an example, one can be found here: https://community.plot.ly/t/dash-0-41-0-released/22131

From this example did anyone figure out how to output to a single .html file? Would love to pass Dash boards to colleagues without requiring server.

vantaka2 commented 5 years ago

@chriddyp - sorry for tagging you on a closed issue! I have the same question as @mneira10 & @guyms above, seems like a few of us are looking for a way to export the html.

Higher up in this thread you mentioned this is possible

This will be possible if you want to just export the app.layout of your app. Nothing new to learn here.

would you mind sharing a example of how to do this? Apologize if this is documented somewhere, I went though a lot of the documentation and have not found it yet.

Thanks again for all your great work!

I asked the same question on the dash community page: https://community.plot.ly/t/export-html-of-dash-app/26754 - I'll be sure to update everyone, if it gets answered or I figure it out.

chriddyp commented 5 years ago

As mentioned above in https://github.com/plotly/dash/issues/266#issuecomment-494532530, exporting to standalone HTML is not officially supported right now, this issue ended up focusing on the clientside callback architecture.

plotly / dash

Clientside Callbacks #266

672

Documentation

Documentation

Documentation