Recommended dashboarding approach

[jlstevens]

This issue summarizes a discussion I had with @jbednar, @philippjfr and @ceball about building dashboard with parambokeh and bokeh server.

We believe we can build dashboards in a fairly flexible and general way using a three level hierarchy:

1. The dashboard object which represents one bokeh application (i.e a tab/webpage). This object has parameters that apply to the entire dashboard (displayed with parambokeh).
2. This dashboard object builds bokeh layouts out of subobjects which also output bokeh layouts. These sub-objects are 'widgeted plots' which combine a set of widgets with a corresponding visualization. These objects have parameters of interest to the user, intended for display with parambokeh: the goal is to build a meaningful unit of functionality combining parambokeh widgets with the associated visualization. I'm not sure what we are calling these components yet.
3. These components in turn own parameterized objects which generate the bokeh plots themselves (i.e via bokeh models or holoviews objects). These components may have many more parameters than are exposed at the dashboard level.

For step 2, you can either use mode='raw' and build a bokeh layout for yourself or you can supply a bokeh model in plots and a view_position. There are a few ways this could be improved:

1. A whitelist of parameter names making it easier to order parameters and drop unwanted parameters from the widgets, instead of relying on precedences.
2. A mechanism to partition the parameters and display them in groups in different locations. e.g you might want parameters 'A' and 'B' on the left, and parameter 'C' on the top of the plot.

I'm not sure how this will work when multiple bokeh models are supplied to plots and to be honest, I haven't checked how it works right now , maybe the widgets are placed in view_position relative to the first element in plots?

One other thing we discussed was the possibility of associating a visualization with corresponding parameters/widgets at the param level and not at the parambokeh level. The idea is that parambokeh would be able to generate a default visualization (with widgets) given a parameterized object and nothing else. You would then be able to customize the details of this visualization by supplying optional information to paramb.

We also discussed how holoviews widgets could fit into this system: perhaps the holoviews widgets would use bokeh widgets and for matplotlib support, we could simply output Div models in a bokeh layout where the Div holds the matplotlib output.

[jbednar]

Thanks for that summary.

I'm not sure what we are calling these components yet.

Good question. WidgetLayout? ParameterizedPlot? There are at least three things that should fit into such a category, as limiting cases: 1. HoloViews_object+widgets, 2. widgets alone (i.e., just a Parameterized, wrapped with Widgets), and 3. HoloViews_object alone (no extra widgets beyond the HoloMap/DynamicMap ones). Each of these consists of zero or more widgets along with zero or more plots, as a unit that can be laid out on a larger page.

We also discussed how holoviews widgets could fit into this system: perhaps the holoviews widgets would use bokeh widgets and for matplotlib support, we could simply output Div models in a bokeh layout where the Div holds the matplotlib output.

Taking a step back, the idea is that the auto-generated HoloMap and DynamicMap widgets should be able to be combined seamlessly with custom widgets, so that the dashboard user doesn't have to know or care that some are automatically synthesized based on dimensions not laid out or overlaid, and others are explicitly created outside of HoloViews itself.

1. The dashboard object which represents one bokeh application (i.e a tab/webpage).

For completeness, can you briefly summarize how any of this interacts with the multiple-tab/dashboard approach we agreed on today? Is there one object that would in turn contain and enclose all three dashboards, and if so, does it have parameters (e.g. buttons that would launch the other dashboards)?

[jbednar]

Also, @philippjfr agreed to add an option to ParamBokeh to allow the ParameterizedPlot (or whatever we call it; currently paramnb.Widgets I guess) to specify a whitelist for parameters to include as widgets, in the specified order. That should make it simpler to control the appearance of these objects in dashboards. I've assigned this issue to him until that task is completed, but after that @philippjfr should remove his assignment and finishing the rest will likely be up to me, @jlstevens, and @ceball.

[jbednar]

Another note: Hopefully the ParameterizedPlot concept can be hierarchical, such that we can lay out a plot with widgets, then combine that with another plot with widgets (and/or just an extra bank of widgets), and so on, to build up chunks of reusable functionality that we can treat as a group. The top-level dashboard may thus end up being the same actual class as the underlying ParameterizedPlot objects, unless the top level really needs some special machinery not present in the lower levels (and even then may end up being a subclass of the ParameterizedPlot class, only adding the top-level-specific functionality.

[jbednar]

Is there one object that would in turn contain and enclose all three dashboards, and if so, does it have parameters (e.g. buttons that would launch the other dashboards)?

Actually, I guess it's the first dashboard that would own all the others, and would be the thing that is launched originally. So, it's worth summarizing how those dashboards would communicate, and how this intersects with any of the above (if it does).

[jlstevens]

For completeness, can you briefly summarize how any of this interacts with the multiple-tab/dashboard approach we agreed on today?

Right now I have a prototype which works but with various important limitations (and some bugs!) so this outline is likely to change.

The simplest thing to communicate between multi-dashboards is to have multiple bokeh applications served by a single bokeh server process, communicating via shared memory (i.e singleton python objects). It is hackish but you can just import some object from a module in two apps and use this object to communicate - this is the approach used in the bokeh spectrogram example. Although simple, this approach has implications when considering threading and there are still some issues I need to work out here.

Is there one object that would in turn contain and enclose all three dashboards, and if so, does it have parameters (e.g. buttons that would launch the other dashboards)?

Again, what I have right now is a prototype (which won't support multi-threading!) so there might be such an object in future. Currently, the object that owns all three applications is a bokeh Server object:

server = Server({'/dash1': Dashboard1()(), '/dash2': Dashboard2()()}, num_procs=1)

Another consequence of this approach is that because this embeds the bokeh server into a regular python script, you launch the dashboards with python app.py not bokeh server app.py.

My other comment is the idea of ParameterizedPlot seems to be something that fits in at the param level. Is there really a need for a new class? How about having BokehModel and Holo parameter types and just used regular Parameterized and ParameterizedFunction? As far as I can tell, all we need from param is a way to collect the normal set of parameters together with objects that have a visual representation.

[jlstevens]

... or to keep things simple a param.Plot parameter that support Bokeh and HoloViews out of the box (if these libraries are available) and that is extensible. I wonder if it could also support matplotlib Figure objects too...

[jbednar]

Is there really a need for a new class?

I don't know. It might just be Parameterized, which would naturally take care of the three things I listed above (only widgets, only plot, both plot and widgets).

[jbednar]

How about having BokehModel and Holo parameter types and just used regular Parameterized and ParameterizedFunction? As far as I can tell, all we need from param is a way to collect the normal set of parameters together with objects that have a visual representation.

I don't want to add any Parameter types to Param that are that specialized. Certain packages can do that for their own purposes (e.g. HoloViews could define one), but I don't think that would be part of Param. Here, I don't think that it needs to be Bokeh or HV or PNG specific. At worst we can take paramnb.View parameters and make them less about paramnb and move that class to param; at best I'm not entirely certain that we need a new Parameter type at all. But let's get to that once we have a good handle on what can be expressed naturally just for Parameters in general.