slow callbacks are all pruned in the presence of a dcc.Interval

[antoinerg]

If the callbacks triggered by a dcc.Interval take longer to complete than the refresh time, they all get pruned and nothing updates.

Example app:

import dash
from dash.dependencies import Input, Output
import dash_core_components as dcc
import dash_html_components as html
import time
app = dash.Dash(__name__)
app.layout = html.Div([
    dcc.Interval(id='i', interval=2000),
    html.Div(id='o')
])
i = 0
@app.callback(Output('o', 'children'), Input('i', 'n_intervals'))
def update_output(n):
    global i
    i += 1
    print([n, i])
    time.sleep(5)
    print([n, i])
    return str([n, i])
app.run_server(debug=True)

dash                 1.20.0
dash-core-components 1.16.0
dash-html-components 1.1.3
dash-renderer        1.9.1
dash-table           4.11.3

Describe the bug

The page never updates: callbacks are all pruned.

From the network tab:

-----
   -----
      -----
         -----

Expected behavior

One callback should eventually complete.

[Marc-Andre-Rivet]

Top of my head I can think of these approaches:

1. We add a mechanism in the renderer informing it that a (component, props) tuple isn't prunable in-flight. Similar to the asyncDecorator flag preventing callbacks prior to the component being rendered ~2. Change the Interval component implementation to make use of the is_loading prop state to either "maybe" tick on interval (not loading) or to only count time that passes while not loading~ (as pointed out this doesn't help anything)
2. We remove pruning in-flight (this is done to avoid renders we know are stale so there would be a performance impact on both renders and additional triggered callbacks)

[Marc-Andre-Rivet]

Confirming this is not a regression. This behavior is the same for all >=1.0.0

[Marc-Andre-Rivet]

Building on option 1 above (https://github.com/plotly/dash/issues/1613#issuecomment-830213596), wondering if this may be viable:

As currently implemented callbacks are "head-prunable" meaning that upon a new callback request, if there's a in-flight request, we'll prune it and make the new callback request the head. In the scenario described above we would make the callback "tail-prunable" in that the in-flight callback wouldn't be affected but the subsequent, not yet in-flight, callback (or tail as we'd never keep more than 2 around) would be the one getting pruned.

This should handle both the scenario with long delays because of i/o, and long delays because of high cpu load (preventing delay accumulation because cpu time for that callback exceeds interval * workers.

For callbacks with multiple inputs, it exposes the fact that being tail/head prunable is not a characteristic of the callback itself but rather a way of processing existing in-flight callback based on a new requested callback trigger. For example:

@app.callback(
  Output(...),
  Input('dccInterval', 'n_intervals'),
  Input('dccInput', 'value')
)
def my_function(...):
  # ...

A dccInterval trigger would tail-prune existing callbacks and a dccInput trigger would head-prune existing callbacks. If there are multiple triggers, the presence of a tail flag takes precedence.

Not certain how subsequent callbacks should behave here. It's easy to come up with a highly variable callback duration scenario where we would execute the initiating callback but none of the subsequent ones due to pruning. This is making me ponder @nicolaskruchten's position on in-flight pruning. Pruning in-flight initiating callbacks still seems fine IMO, but there's definitely space for interpretation when it comes subsequent callbacks in the chain -- how much do we value performance vs. consistent state. Without consideration to subsequent callbacks, top of my head, there's still potential inconsistency / pruning issues with no_update, PreventUpdate, and at the very least when using as state one of the output props.

[theavgjojo]

Hey all, I'm running the latest dash:

dash                 2.8.1
dash-core-components 2.0.0
dash-html-components 2.0.0
dash-table           5.0.0
Flask                2.2.3
Werkzeug             2.2.3

I most commonly run into the same behavior with intervals, switching in multi-page apps, and receiving user inputs with longer callbacks. I wrote a MWE for a new issue but fortunately found this stale one which I think is the cause. Still, below is some code that can result in pruning if an interval fires within the 1s button callback handling:

import time
import dash
from dash import Dash, html, Input, Output, dcc

app = Dash()

app.layout = html.Div([
    html.Button('Test', id='button_test'),
    html.Div(children='', id='div_output'),
    dcc.Interval(
        id='interval_test',
        interval=3000,
    ),
])

@app.callback(
    output={
        'div_output__children': Output('div_output', 'children'),
    },
    inputs={
        'n_clicks': Input('button_test', 'n_clicks'),
        'n_intervals': Input('interval_test', 'n_intervals'),
    },
)
def test(n_clicks, n_intervals):
    output = {}
    output['div_output__children'] = dash.no_update
    ctx = dash.callback_context
    if ctx.triggered and n_clicks:
        for context_event in ctx.triggered:
            input_id, input_property = context_event['prop_id'].split('.', 1)
            if input_id == 'button_test':
                time.sleep(1)
                print(n_clicks)
                output['div_output__children'] = str(n_clicks)
    return output

if __name__ == "__main__":
    app.run_server(debug=True)

You'll see the POST response come back with something as follows:

{"multi":true,"response":{"div_output":{"children":"5"}}}

However, if the interval fires during the callback, it will return 204 NO CONTENT and the div will not get updated with 5. The interval can return before or after the button handler finishes. As long as the interval triggers the callback during the button-triggered callback the update will be pruned.

Are there any strategies or good design patterns to work around this behavior? Given that the pruning happens in the client JS I haven't found much to control to workaround the issue.