Race condition when updating data source and x_range

smaier-by commented 6 years ago

Software version info

bokeh server on Debian 8 (vagrant virtual machine)
Chrome 62.0.3202.94 (on MAC OS)
bokeh 0.12.7, 0.12.9, 0.12.10, 0.12.11 (with bokeh serve)
python 2.7

installed python packages

backports-abc==0.5
bokeh==0.12.11
certifi==2017.11.5
futures==3.2.0
Jinja2==2.10
MarkupSafe==1.0
numpy==1.13.3
pandas==0.21.0
python-dateutil==2.6.1
pytz==2017.3
PyYAML==3.12
singledispatch==3.4.0.3
six==1.11.0
tornado==4.5.2

Expected behavior vs. observed behavior

Expected behavior I would like to update both a data source and an x_range of some plot in a on_value callback on a TextInput. If a data source update and an update of the x_range are done sequentially in the update function, I would expect the displayed x_range to be determined by the values to which x_range.start and x_range.end were explicitly set.
Observed behavior Sometimes, the displayed x_range behaves as if the x_range had not been set at all. If a sufficiently long sleep is performed between the data source update and the x_range update, the x_rangeis then displayed in the browser as expected. There must hence be a race condition (I would assume on the JavaScript side).

Minimal code example

import pandas as pd

from bokeh.plotting import figure, curdoc
from bokeh.layouts import column
from bokeh.models.sources import ColumnDataSource
from bokeh.models.widgets import TextInput

from time import sleep

def dashboard():
    all_data = pd.DataFrame({'y': [1, 2, 3] * 5,
                             'x': range(15)})

    data_source = ColumnDataSource(all_data)

    selector = TextInput()

    plot = figure(plot_height=300, plot_width=1000, title='Example')
    plot.line('x', 'y', source=data_source)

    def updata_data(attr_name, old, new):
        """
        If this sleep is removed or if the sleeping time is reduced to 1 ms,
        the axis update sometimes fails.
        """
        new_data = ColumnDataSource.from_df(all_data.query("x >= {}".format(new)))
        data_source.data.update(**new_data)
        #sleep(0.2)
        plot.x_range.start = int(new) - 1.5
        plot.x_range.end = 18.5

    selector.on_change('value', updata_data)

    curdoc().add_root(column(selector, plot))

dashboard()

Stack traceback and/or browser JavaScript console output

Bokeh server


(bokeh_experiments) vagrant@jessie-test:~$ bokeh serve --port 5567 --show minimal_example.py

2017-12-05 17:34:13,953 Starting Bokeh server version 0.12.11 (running on Tornado 4.5.2) 2017-12-05 17:34:13,955 Bokeh app running at: http://localhost:5567/minimal_example 2017-12-05 17:34:13,955 Starting Bokeh server with process id: 874 2017-12-05 17:34:21,149 200 GET /static/js/bokeh-gl.min.js.map (10.0.2.2) 19.66ms 2017-12-05 17:34:21,155 200 GET /static/js/bokeh-widgets.min.js.map (10.0.2.2) 32.05ms 2017-12-05 17:34:21,250 200 GET /static/js/bokeh-tables.min.js.map (10.0.2.2) 125.31ms 2017-12-05 17:34:21,458 200 GET /static/js/bokeh.min.js.map (10.0.2.2) 346.86ms 2017-12-05 17:34:23,124 200 GET /minimal_example (10.0.2.2) 77.84ms 2017-12-05 17:34:23,388 101 GET /minimal_example/ws?bokeh-protocol-version=1.0&bokeh-session-id=4e2sdAyWmhyJ29VstqB7R47el0NfDQYXsBifEOSlNIoB (10.0.2.2) 0.48ms 2017-12-05 17:34:23,388 WebSocket connection opened 2017-12-05 17:34:23,389 ServerConnection created


- Browser console

[bokeh] setting log level to: 'info' connection.js:194 [bokeh] Websocket connection 0 is now open widget_box.js:108 [bokeh] WidgetBox mode is fixed, but no width specified. Using default of 300. document.js:168 [bokeh] document idle at 76 ms embed.js:291 Bokeh items were rendered successfully


#### Screenshots or screencasts of the bug in action

- Desired behavior
  _Obtained by uncommenting the sleep in the above code example and typing 8 in the_ `TextInput`
<img width="1081" alt="desired_behavior" src="https://user-images.githubusercontent.com/30986601/33621110-0f8a73f4-d9ea-11e7-82d2-ff5603a4268f.png">

- Actual behavior (sometimes)
  _Obtained by commenting/removing the sleep in the above code example and typing 8 in the_ `TextInput`
<img width="1062" alt="update_x_range_fails" src="https://user-images.githubusercontent.com/30986601/33621215-57f3d888-d9ea-11e7-9dd1-88827954ebea.png">

bryevdv commented 6 years ago

Since you are setting range extents manually anyway, I would suggest doing that from the start instead of having the default (auto-ranging) DataRange1d Changing this line in your code results in expected behavior:

plot = figure(plot_height=300, plot_width=1000, title='Example', x_range=(9,14))

The issue is as you note, almost certainly a race between the explicit overriding of start and end of the default DataRange1d and the automatic range computation that DataRange1d performs normally. Unfortunately I am not sure this will be able to be prioritized in the immediate term, especially given that there is a reasonable workaround.

smaier-by commented 6 years ago

Thanks @bryevdv! That worked.

bryevdv commented 6 years ago

I think this might be a dupe of https://github.com/bokeh/bokeh/issues/4014

EfremBraun commented 3 years ago

I have a slightly different situation for which I don't see a workaround. I'd like some changes by the user to not result in any changes to the x_axis, and some changes by the user to auto-scale the x-axis. Based on #8171, I'd expect this to work:

import pandas as pd

from bokeh.plotting import figure, curdoc
from bokeh.layouts import column
from bokeh.models.sources import ColumnDataSource
from bokeh.models.widgets import TextInput

def dashboard():
    all_data = pd.DataFrame({'y': [1, 2, 3] * 5,
                             'x': range(15)})

    data_source = ColumnDataSource(all_data)

    selector_keep = TextInput(title='keep scale')
    selector_scal = TextInput(title='autoscale')

    plot = figure(plot_height=300, plot_width=1000, title='Example')
    plot.line('x', 'y', source=data_source)

    def updata_data_keep(attr_name, old, new):
        old_start = plot.x_range.start
        old_end = plot.x_range.end
        new_data = ColumnDataSource.from_df(all_data.query("x >= {}".format(new)))
        data_source.data.update(**new_data)
        plot.x_range.start = old_start
        plot.x_range.end = old_end

    def updata_data_scal(attr_name, old, new):
        new_data = ColumnDataSource.from_df(all_data.query("x >= {}".format(new)))
        data_source.data.update(**new_data)
        plot.x_range.start = None
        plot.x_range.end = None

    selector_keep.on_change('value', updata_data_keep)
    selector_scal.on_change('value', updata_data_scal)

    curdoc().add_root(column(selector_keep, selector_scal, plot))

dashboard()

However, the behavior is not at all what I'd expect. If the graph is new or the "Reset" button has just been pushed, the two TextInput widgets both result in auto-scaling (presumably because plot.x_range.start and plot.x_range.end remain at None, but I'm not sure how to fix that). And once the user moves the graph, the two TextInput widgets both result in the the scales being kept (I don't know why; I would think that setting the plot.x_range.startand plot.x_range.end to None would trigger an autoscale based on #871).

bryevdv commented 3 years ago

TBH trying to take a quick look I am not seeing where setting post-initialization values for start or end would trigger an update. This may actually be just be a case of missing event plumbing.

That said, approximately a dozen different little features multiplicatively converge on data ranges, making it one of the most complicated and convoluted parts of the entire codebase. I may be at the point of thinking we just have to start over for data-ranges in 3.x in order to find ways to separate and simplify things, so that one class is not so drastically overloaded. cc @mattpap

bryevdv commented 2 years ago

Noting there is a simpler reproducer in https://github.com/bokeh/bokeh/issues/11980

bokeh / bokeh