add callbacks dynamically

[endremborza]

Hi!

this is somewhat of a mixture of a feature request and a bug report. here is my example:

import dash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output, State
import json
import os

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']
app = dash.Dash(__name__, external_stylesheets=external_stylesheets)
app.config.suppress_callback_exceptions=True

try:
    os.remove('cities.json')
except OSError:
    pass

def make_cb(city):
    @app.callback(Output('%s-pushed' % city, 'children'),
                  [Input('%s-but' % city, 'n_clicks')],
                  [State('%s-but' % city, 'children')])
    def fing(nc,cval):
        if nc:
            return cval
        return ''

def get_layout():

    try:
        cities = json.load(open('cities.json'))
    except FileNotFoundError:
        cities = []

    return html.Div([
                dcc.Input(id='input-1', type='text', value='Montréal'),
                html.Button(id='submit-button', n_clicks=0, children='Submit'),
                html.Div(id='pushed-city'),
                html.Div([html.Div([html.Button(c,id='%s-but' % c),
                                    html.Div(id='%s-pushed' % c)])
                          for c in cities],id='output-state')
            ])

app.layout = get_layout

@app.callback(Output('output-state', 'children'),
              [Input('submit-button', 'n_clicks')],
              [State('input-1', 'value')])
def update_output(n_clicks, input1):
    try:
        cities = set(json.load(open('cities.json')))
    except FileNotFoundError:
        cities = set()

    if input1 not in cities and input1 is not None:
        cities.add(input1)
        cities = list(cities)
        json.dump(cities,open('cities.json','w'))
        make_cb(input1)
    print(app.callback_map)
    return [html.Div([html.Button(c,id='%s-but' % c),
                                    html.Div(id='%s-pushed' % c)])
                          for c in cities]

if __name__ == '__main__':
    app.run_server(debug=True)

this is a simple app based on the documentation, that allows the user to add interactive elements, with some caching. There are two issues. The first one I don't understand, the second one I do.

1. If I click the submit button, to add a city, everything appears fine, the print(app.callback_map) line shows me that the callback has been registered, i can see everything in the browser, however the callback does not become active, until I refresh the browser.

2. If I use a proper server like gunicorn not the flask development server, with parallel processes, the callback_map dictionary is not shared, so the whole thing breaks down, and the callbacks basically don't work.

Now, I'm quite sure that 1, has some easy explanation that I'm just a little too tired to figure out, I wrote this whole thing because of 2,

I also hacked together a solution for myself for the second one, that allows for declaring the callback map as a shared key-value store, using redis. If anyone's interested, I can post a PR. As far as i can see, only __iter__, __getitem__, __setitem__ and items() are utilized from the callback_map.

I looked through #577 #475 #373 this and this. Based on these, creating a shared key-value store for the callbacks is a well isolated issue that fits the main arch of these topics.

[endremborza]

If I click the submit button, to add a city, everything appears fine, the print(app.callback_map) line shows me that the callback has been registered, i can see everything in the browser, however the callback does not become active, until I refresh the browser.

ok, so now I understand that this is because the _dash-dependencies endpoint is only called on page refresh. I really hope this can be solved as well somehow, but it is a different issue from 2, which would be my main point here. (also, refreshing the dependencies as a callback is added is probably higher priority)

[alexcjohnson]

Hi @endremborza - thanks for continuing this discussion, and sorry for taking so long to respond.

The other piece that complicates this further is multiple users of the app - so there can be multiple server processes that would all need to know about these dynamic callbacks, but there can also be multiple users, all of whom could have different callbacks defined based on their actions.

To make this kind of dynamically-added callbacks work with multiple users, we would somehow need to inform the server which callbacks to use for each request, and have those callbacks shared between server processes. Presumably that would be some sort of uid sent with every request, that gets matched up with the appropriate callbacks by the server.

Something like this could be done, but I think a system like #475 will serve the same role with a simpler structure that allows everything to be defined upfront.

[endremborza]

Yes, I have just recently come across this, I suppose some session handling would be needed for this, I'll look into it.

Also, the only problem I have with #475 is just the fact that it's predefined, which I suspect must have some restrictions on what interactions a user can add.

Anyway, if I don't find a simple enough solution to handling different users, I think I'll try reviving #475

[alexcjohnson]

Also, the only problem I have with #475 is just the fact that it's predefined, which I suspect must have some restrictions on what interactions a user can add.

Possible - that said before we start adding session handling I think it would be worth coming up with some specific tricky use cases and seeing if that suspicion is borne out or if in fact it can cover all the cases we can come up with.