Checking pattern matching compatibility with ServersideOutput

[DeKhaos]

Hi, since the document didn't list down any limitations for interaction between ServersideOutput and pattern matching, so I tried to do some implementations and face some issues, as follows:

• ServersideOutput doesn't allow wildcard ALL when trying to save multiple dataframe to a list of dcc.Store

• State of ServersideOutput inside a callback when using pattern matching doesn't return the relevant complex object but an random string like an id to cache or something. According to #192, MATCH pattern does return an object to work with when match is found.

Would be nice if anyone can give me an example on how to use pattern matching with ServersideOutput.

Sample code (some comment is also included):

from dash_extensions.enrich import DashProxy,Input,Output,State,ServersideOutput,\
    ServersideOutputTransform,ALL,MATCH,dcc
from dash import html,ctx
import dash_bootstrap_components as dbc
import plotly.express as px

app = DashProxy(__name__,
           external_stylesheets=[dbc.themes.CYBORG,
                                 dbc.icons.BOOTSTRAP],
           transforms=[ServersideOutputTransform()])

app.layout = html.Div([
    dcc.Store(id={'index':0}),
    dcc.Store(id={'index':1}),
    dcc.Store(id={'index':2}),

    dbc.Button('Load data',id='load_button'),
    dbc.Button('Show data',id='show_button'),
    html.Br(),
    dbc.Label("Display 1:"),
    html.Div(id={'type':'display','index':0}),
    dbc.Label("Display 2:"),
    html.Div(id={'type':'display','index':1}),
    dbc.Label("Display 3:"),
    html.Div(id={'type':'display','index':2})
])

@app.callback(
    # 'ALL' pattern match doesn't work if DataFrame is not serialized to JSON
    ServersideOutput({'index':ALL},'data'),

    #List all output doesn't cause error 
    # [ServersideOutput({'index':0},'data'),
    # ServersideOutput({'index':1},'data'),
    # ServersideOutput({'index':2},'data')],

    Input('load_button','n_clicks'),
    prevent_initial_call=True
    )
def load_data(click):
    #ctx does print out the full output list
    print(ctx.outputs_list) 

    #this doesn't cause error if I use ALL pattern-match for ServersideOutput
    #but doesn't serve the purpose of ServersideOutput to store complex objects
    # return [px.data.carshare().head(3).to_dict(),
    #         px.data.election().head(3).to_dict(),
    #         px.data.iris().head(3).to_dict()]

    #this cause error if I use ALL pattern-match for ServersideOutput
    #but work ok if list down individual ServersideOutput
    return [px.data.carshare().head(3),
            px.data.election().head(3),
            px.data.iris().head(3)]

@app.callback(
    Output({'type':'display','index':MATCH},'children'),
    Input('show_button','n_clicks'),
    State({'index':MATCH},'data'),
    prevent_initial_call=True
    )
def show_output(click,data): 
    #in case data was stored successful to dcc.Store (using ServersideOutput),
    #'data' only return some random id to cache or something and not the desired 
    #DataFrame|dictionary
    return str(data)

if __name__ == '__main__':
    app.run(debug=True)

Environment: dash 2.7.0 dash-extensions 0.1.7

[jonasvdd]

Hi @emilhe thank you for your amazing work!

I experience this issue as well; could you provide me some hints where I should start looking in your codebase to resolve this?

Kind regards, Jonas

[DeKhaos]

Hi @jonasvdd, if you're interested in a temporary solution while waiting for an official fix I can help you with that.

Solving the compatibility with ALL pattern matching might take some further digging, but I found a way to work with MATCH keyword.

ServersideOutput directly stores the object as cache to either local disk|Redis instead of de-serialization, which make it much faster. It uses flask_caching.backends as the base so you need a key to retrieved the cache. As you can guess, the key is the random string which I snapped shot above :smiley:. If the key is all we need then retrieving it should be easy. In case you want to dig deeper, the problem should be inside sub-module enrich.

This sample code should help you.

from dash_extensions.enrich import DashProxy,Input,Output,State,ServersideOutput,\
    ServersideOutputTransform,ALL,MATCH,dcc
from dash import html,ctx
import dash_bootstrap_components as dbc
import plotly.express as px
from dash_extensions.enrich import FileSystemStore
from dash.dash_table import DataTable
app = DashProxy(__name__,
           external_stylesheets=[dbc.themes.CYBORG,
                                 dbc.icons.BOOTSTRAP],
           transforms=[ServersideOutputTransform()])

app.layout = html.Div([
    dcc.Store(id={'index':0}),
    dcc.Store(id={'index':1}),
    dcc.Store(id={'index':2}),

    dbc.Button('Load data',id='load_button'),
    dbc.Button('Show data',id='show_button'),
    html.Br(),
    dbc.Label("Display 1:"),
    DataTable(id={'type':'display','index':0}),
    dbc.Label("Display 2:"),
    DataTable(id={'type':'display','index':1}),
    dbc.Label("Display 3:"),
    DataTable(id={'type':'display','index':2})
])

@app.callback(
    #avoid using ALL keyword,list down individual output if necessary or use other component to update the outputs
    [ServersideOutput({'index':0},'data'),
    ServersideOutput({'index':1},'data'),
    ServersideOutput({'index':2},'data')],
    Input('load_button','n_clicks'),
    prevent_initial_call=True
    )
def load_data(click):
    return [px.data.carshare().head(3),
            px.data.election().head(3),
            px.data.iris().head(3)]

@app.callback(
    Output({'type':'display','index':MATCH},'data'),
    Output({'type':'display','index':MATCH},'columns'),
    Input('show_button','n_clicks'),
    State({'index':MATCH},'data'),
    prevent_initial_call=True
    )
def show_output(click,data):

    #refer to storage location on local disk or Redis
    directory = FileSystemStore() #add path to cache location if not default

    df = directory.get(data)#retrieve dataframe from cache with key

    return df.to_dict('records'),[{"name": i, "id": i} for i in df.columns]

if __name__ == '__main__':
    app.run(debug=True)

[emilhe]

It seems to be an issue with the unpacking. The code has gotten a bit complex after Dash introduced flexible callback signatures, so it might require a bit of effort to fix it.