Open philippjfr opened 1 year ago
Thanks for that post, thinking about all of that is timely! I haven't come up with any solution yet, still thinking (it's a lot!), I just have for now a couple of precision/suggestions.
param.Parameter
is indeed the basis of all of the reactivity, I read somewhere that sort of thing can be called reactive dependency which I quite liked. I also like to think about it as an input to a computation graph.
There are a few differences in how updates are handled:
param.Parameter
: updating the value doesn't trigger by default, you need to register callbacks (with e.g. .param.watch
, @param.depends
)param.bind
and @param.depends
: updating reactive dependencies doesn't trigger by default, you need watch=True
param.reactive
(formerly hvplot.interactive
): updating reactive dependencies triggers by default, there's no way (yet? is that even desirable?) to make it just a "declarative reactive" (yuk) expressionSo param.bind
and @param.depends
are purely declarative by default, they just express that a function/method depends on some other reactive inputs (they are really meant to be leveraged by some other system, e.g. a GUI library like Param, and we should probably document how). Without watch=True
, can these be even called reactive? They don't react much! With watch=True
(I dislike pn.bind(..., watch=True)
) they effectively become reactive.
I feel that there's a need to align somehow @param.depends
, param.bind
and param.reactive
on this front. .param.watch
is low-level and clear enough to me.
The only think I'd like to say for now is that we have seen that the gap between Panel and HoloViews/hvPlot is big enough for users to have a hard time understanding, or just getting to know about, DynamicMap
s, leading to sub-optimal data viz apps and spreading of an anti-pattern. So whatever we do should not make that gap bigger, or make it easier for Panel users to do the wrong thing.
Good summary of the situation but I don't see how we can come up with yet another (supposedly more unified) API without making the situation worse. Plenty of code exists out there using all the different approaches and often mixing them together and so we can't get rid of anything without breaking a lot of existing code. While it would be nice to have one way to do everything, in practice I expect this would just add one more way of doing things to the mix.
For now I would point out that in your example above, I would use the following which is also supported:
def function(x, y):
return hv.Points([x, y])
tap = Tap()
hv.DynamicMap(function, streams=dict(x=tap.param.x, y=tap.param.y))
I would also say that I agree that passing by reference is probably the best approach - of course that wasn't an option when a lot of these APIs first evolved which is why things became so confusing. And of course, passing by reference is also tricky at times e.g having to use strings to reference parameters in param decorators.
It's encouraging that this discussion is happening and I hope it leads to cleaning up the reactive story. Creating reactive dependencies always trips me up, especially when I'm working across packages.
Here's what I think should be possible at a high level with pseudocode:
I want to naturally use a widget-like object as an input, without fumbling with streams
, DynamicMap
, apply
, opts
, watch
, bind
, 🤯 . Additionally, I want to use valid interactions on one object as dynamic and explicit input to another object, and I don't want to have to use any 'terminating' methods.
slider = Slider(start=1, end=10)
scatter = Scatter(data, marker_size=slider)
table = Table(reactive(data).sel(x=scatter.tap.x))
The story should be that everything is widget-like - interactive and subscribable.
In that example marker_size
is not something that would ever be supported by hv.Scatter
. However, this suggestion is something that I think would be entirely appropriate for hvplot
to handle.
Instead of trying to have a single story for reactivity everywhere, I would focus on having a single, powerful story for hvplot
if we want it to be the entry point for most people (at least when it comes to plotting).
hvPlot supports passing widgets as arguments. Although I'm pretty sure months ago we said that it should be deprecated (yet another way...): https://hvplot.holoviz.org/user_guide/Widgets.html#using-widgets-as-arguments
Also works if you first make it interactive, in which case the widget is displayed automatically (yet another way!):
I liked Demetris' example, I think we should come up with many more examples and see how we'd like to write them (@droumis in that particular example, I wonder .sel(x=scatter.tap.x
would work, you need to define a hit radius right?).
@maximlt.. hmm maybe instead of tap, a more useful example would be the following, where the .selection.x
is the x index of the data currently selected with a selection-type tool.
slider = Slider(start=1, end=10)
scatter = Scatter(data, marker_size=slider)
table = Table(reactive(data).sel(x=scatter.selection.x))
I'm sure there are a million other ways to write this:
import pandas as pd
import numpy as np
import hvplot.pandas
import holoviews as hv
import panel as pn
pn.extension('tabulator')
df = pd.DataFrame(np.random.random((10, 3)), columns=list('abc'))
slider = pn.widgets.IntSlider(value=1, start=1, end=30, step=5)
scatter = df.hvplot.scatter('a', 'b', tools=['box_select'])
pn.Column(
slider,
scatter.apply.opts(size=slider),
df.interactive().iloc[hv.streams.Selection1D(source=scatter).param.index]
)
Not so many lines of code compared to Demetris' example. Probably quite representative of normal users code, mixing concepts from panel/holoviews/hvplot and having to visit all these sites and copy/paste stuff to get something working.
@jbednar and I have discussed the idea of accessing streams via an accessor on HoloViews components for a long time, i.e. <element>.events.selection.indexes
would automatically create a Selection1D
stream attach it on the element and then make it available going forward. At some point I even had a prototype. I'm strongly in favor of this and it would be a very small amount of effort.
As for the overall discussion about HoloViews and reactivity, I do think to some extent that getting rid of DynamicMap
and making individual elements reactive is only way we're going to get to a sensible "reactive" story in HoloViews. Except I may tweak your suggestion to keep the options separate: hv.Scatter(<reactive_data>).opts(size=widget)
. My feeling on DynamicMap
is simply that it's an additional, very confusing layer that gets in the way of having a declarative and reactive specification of a plot. Writing callbacks (even if they are declarative/reactive callbacks as opposed to imperative callbacks) is a major barrier to readability and while hvPlot can largely abstract away the API, you still end up returning a DynamicMap
to the user which is very opaque and confusing in most cases. This certainly would be quite a long term goal though.
hvPlot supports passing widgets as arguments. Although I'm pretty sure months ago we said that it should be deprecated (yet another way...)
The main concern there was that passing widgets could change the return type from a HoloViews component to a Panel component. This indeed should not be allowed, i.e. we should only allow passing widgets for options that do not require Panel to fully re-render the element type.
Just going to drop this here if it's helpful; my attempt in enumerating all the ways to interact with HoloViz objects:
I would focus on having a single, powerful story for hvplot if we want it to be the entry point for most people (at least when it comes to plotting).
I don't think that necessarily addresses anything; hvPlot returns a DynamicMap in many cases, and if people need to work with DynamicMaps, unless the story around DynamicMaps and streams is cleaned up, the complexity will leak through to the end-user's abstraction level.
Once this gets implemented, it'd would be a nice test to rewrite the following notebook to get a feel for the new implementation: https://pydeas.readthedocs.io/en/latest/holoviz_interactions/tips_and_tricks.html
The notebook uses a variety of interactions through DynamicMap
, streams
, bind
, param.watch
, apply.opts
.
Across HoloViz we have been slowly converging on a powerful set of reactive constructs that make it possible to build reactive data pipelines and UI components. In particular we have a set of three core reactive objects that we support.
Reactive objects
Reactive objects represent a reference to a value that can be updated dynamically and then drive reactive computations. Currently we have three such core concepts:
param.Parameter
: A parameter is the basis for all reactive computations and represents a reference to a value that can change and notify downstream consumers of that value.param.bind
(formerlypanel.bind
): A function or generator that can update dynamically. I propose we call these reactive functions/generators.param.reactive
(formerlyhvplot.interactive
): A proxy for an object that can update dynamically. I propose we call these reactive expressions.In addition to these core constructs we treat certain other objects as dynamic references.
panel.Widget
: A widget is treated as equivalent to it'svalue
parameteripywidgets.Widget
: An ipywidget is wrapped in a Parameterized which reflects it'svalue
traitlet.Reactive objects are useful on their own and can be used to drive computation and logic OR they can be used as inputs to a visual pipeline to be rendered by Panel or HoloViews.
Renderers
Panel and HoloViews are powerful rendering engines for plots and UI elements that (imperfectly) support dynamic references. In particular we have a variety of ways to take the reactive constructs and turn those into displayable components. Before we cover all the ways let's summarize the potential consumers of our reactive objects:
DynamicMap
: A DynamicMap can consume the output of a function and render HoloViews objects rendered by it. They can be generated as follows.The problem we face here is that these objects are pretty inconsistent in the way they handle the reactive objects that can serve as their inputs.
HoloViews
Let's start for example with the
DynamicMap
. It long preceeds most of the reactive concepts and therefore has multiple ways of binding a dynamic reference:Stream
sHoloViews
Stream
s preceed all reactive components in the HoloViz ecosystem. They hold parameters and have a way of notifyingsubscribers
of the changes. At this point this is extremely duplicative of all the other mechanisms we have in HoloViz and does not play well with any of it. To construct aDynamicMap
from a stream you do the following:This binds the parameters of the
Tap
stream to the keyword arguments of the function.bind
A much more explicit way to bind parameters to a
DynamicMap
isbind
:This is much closer to the way the rest of HoloViz works.
Element.apply
HoloViews also offers a
.apply
method which allows taking a staticElement
and dynamically applying options or other operations to it, e.g.:Creates a DynamicMap to dynamically apply the size.
hvplot.interactive
(param.reactive
)hvPlot
.interactive
orparam.reactive
pipelines can terminate in aDynamicMap
by calling.holoviews()
.Summary
There are a number of ways of creating reactively updating objects (
DynamicMap
) in HoloViews. However there is little consistency:streams
: Old API completely unrelated to everything else we do in HoloVizbind
: Good that HoloViews can consume these functions but why can't we create a DynamicMap directly from a parameter or aparam.reactive
pipeline..apply
: This is great, we can directly consume any reactive objects as arguments.hvplot.interactive
: Terminating methods are confusing and we should decide whether we preferhv.DynamicMap(<reactive_object>)
or a terminating method (or maybe both).Panel
Panel now has two main ways to respond to reactive changes:
pn.<component>(<ref>)
pn.<component>(parameter=<ref>)
ParamFunction
wrapperpn.panel(param.bind(...))
(orpn.param.ParamFunction(param.bind(...))
)This is more consistent than HoloViews but there are still gaps here.
Summary
We have a number of ways of constructing reactively updating UI components and we are not consistent about which we support:
Passing by reference
Passing a reactive object by reference is probably the cleanest and most readable approach, i.e.
pn.pane.Markdown(str_param, height=int_param)
is very readable, similarlyhv.Curve(...).apply.opts(line_width=widget)
is too. The only thing missing here is the ability to update a reference if needed, e.g.panel_obj.param.update(width=new_widget)
should unbind any old references and bind the new reference.Terminating methods vs Constructors
We are highly inconsistent in our support for terminating methods and passing of references to a constructor, e.g.
pn.panel(param.reactive(...))
andparam.reactive(...).panel()
will both be supported buthv.DynamicMap(param.reactive(...))
orparam.bind(...).panel()
will not as of today.Actions
We need to come up with a way forward here that unifies our stories and makes the APIs symmetric. Some important questions to address before then are:
DynamicMap
and Panel objects) to accept all reactive objects?@holoviz-developers please think about this very deeply because we are fairly close to finally unifying our "reactive" story but there's a number of clear gaps that stop us from telling the story cleanly.