holoviz / panel

Panel: The powerful data exploration & web app framework for Python
https://panel.holoviz.org
BSD 3-Clause "New" or "Revised" License
4.64k stars 504 forks source link

Add one or more libraries of web components to enable creating industrial quality apps in Panel #1120

Closed MarcSkovMadsen closed 3 years ago

MarcSkovMadsen commented 4 years ago

My Pain - Modern looking analytics in Panel

I would like to create awesome apps in Python and Panel if possible. I think most of the things are there

But the app you can create does not look consistent and modern.

One way to address this is using the Template system which is on the Roadmap to document better. See this feature request

Another solution would be to add one or more web component libraries to Panel.

Solution

Add one or more web components libraries like

For more see https://blog.bitsrc.io/9-web-component-ui-libraries-you-should-know-in-2019-9d4476c3f103

Examples

Material

image

Vaadin

image

Predix

image

MarcSkovMadsen commented 4 years ago

Web Components are just custom HTML components and are easy to use. One example is

https://www.predix-ui.com/#/elements/px-app-nav

image

<px-app-nav
items='[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]'
selected-route='["home"]'
selected-meta='{"item":{"label":"Home","id":"home","icon":"px-fea:home"},"path":[{"label":"Home","id":"home","icon":"px-fea:home"}],"route":["home"],"parent":null,"children":[],"siblings":[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]}'>
</px-app-nav>

I think (but don't know for sure) that they would be very easy to develop and use in Panel.

My questions for using Web Components in Panel are

MarcSkovMadsen commented 4 years ago

One of the requirements (or hopes) that I would have is that I could configure which set of components to use like

pn.config.components="material"

and then that set of components would be used when we are displaying parameters from param with for example pn.Param(power_curve).

The original components would be accessible using the string "panel" or "classic".

bryevdv commented 4 years ago

Do they work well with the Bokeh layout engine?

I'd actually love to start finding ways to integrate components without using our layout (for DOM elements).

An extremely unformed idea about how to do this, and to streamline integrations so that complicated extensions are not needed, is this: Develop a set of "virtual widgets" for Bokeh that are not tied to any particular implementation (as is the case now with all the built-in widgets), and do not participate in layout (also the case). They would just expose hooks that would just form one end of a standard "protocol" for each widget. Then all that is needed is a single bit of JS adapter code that can connect the signals and events from your arbitrary slider (that you position and configure however you like, independent of Bokeh), to the standard "slider hooks" of the Bokeh virtual slider.

Now then, higher level libraries like panel might want to to more to help arrange components in templates, might provide sets of standard adapters and use them automatically, etc but this mechanism would drastically lower the burden of creating extensions (at least for the set of virtual widgets) which is good for everyone.

MarcSkovMadsen commented 4 years ago

Thanks @bryevdv . I'm sorry but I have yet to learn how bokeh models and the layout engine really works. Is there a way to add a new model that does not use the bokeh layout engine?

My simple understanding is that if I could avoid having the px-app-nav tag wrapped in a bokeh div then everything would work much better when adding, styling and laying out web components.

bryevdv commented 4 years ago

I'm sorry but I have yet to learn how bokeh models and the layout engine really works. Is there a way to add a new model that does not use the bokeh layout engine?

No, currently, not really at all. Affording that capability is essentially exactly what I am proposing. This does not exist yet, but could. I want to split the "purely event/data" aspect of widgets away from any styling and layout. So that widgets that exist on a page, completely independently of Bokeh, however they got there, can be wired up to Bokeh events and callbacks easily.

I don't want to hijack this issue with this one Bokeh idea, but I realized this is the second time I've had this idea, and this time around I think it is clear that something like this is need so that Bokeh can play to its strengths (sophisticated highly flexible plotting, and interactive eventing in both JS and Python), and start to distance itself from its main weakness (working against the grain of typical web dev, re: layout and styling). So, in the spirit of that, here is a research legwork task that would be great to set the stage for an eventual issue on the Bokeh repo, if someone has the bandwidth for it:

Here's a simple partial examples:

Then to include any Toggle at all from any component library, you provide an adapter that can update that kind of slider when the Bokeh state changes, or conversely, can update Bokeh "virtual Toggle" state when the real widget changes. Maybe think of this as a proposal for a shadow DOM for Bokeh. But the point is these adapters would be much simpler to create than current Bokeh extensions, which require alot of obscure and indecipherable boilerplate code and knowing how to plug in to Bokeh's layout (not easy).

Edit: I'm imagining something like this:

myslider = VirtualCheckbox(adapter="""
    // JS code that:
    // * finds a Vaadin checkbox (e.g) on the page
    // * connects its events to update VirtualCheckbox properties
    // * connects VirtualCheckbox properties to update the Vaadin checkbox
""")

No new Python class to make. No complicated TS model or view classes to make. Just a block of JS code that knows how to wire some specific Checkbox widget up to the generic data/events that a Bokeh VirtualCheckbox has. Any layout, styling, etc. is all done outside Bokeh, in a template, using "standard" web dev. This is where e.g. Panel could really add value by helping to take care of that part and have a collection of adapters for different component libraries.

The other value of this is that, while BokehJS internals are even now still in flux, a VirtualWidget API could be much more stable, making the value of adapters much more durable than full custom extensions (which are easy to break between releases)

MarcSkovMadsen commented 4 years ago

Thanks @bryevdv

My thoughts have more been along the lines of creating a base Panel WebComponentWidget and WebComponentLayout (if possible) where the developer just needs to specify the tag, attributes and which attributes to bind to.

To make my thoughts more concrete I've written the below illustration of some of the ideas I have. It only illustrates the idea of only having to implement/ configure a Panel widget and then the rest is created automatically.

I don't know if it is possible to compile the Bokeh model once and for all. The alternative to creating a real class is creating the code string of the class.

import param
from panel.widgets.base import Widget
from bokeh.models.layouts import HTMLBox
from bokeh.core import properties

class WebComponentWidget(Widget):
    _tag = ""

    def __init__(self, **params):
        self._widget_type = self._get_bokeh_py_model()

        super().__init__(**params)

    @staticmethod
    def _get_bokeh_property(parameter):
        if isinstance(parameter, param.String):
            return properties.String(default=parameter.default)
        else:
            raise NotImplementedError

    def _get_bokeh_py_model(self):
        class_name = type(self).__name__ + "BKModel"
        bases = (object,)  # bases = (HTMLBox,) raises KeyError

        attributes = {}
        self_keys = set(self.param.objects().keys())
        widget_keys = set(Widget.param.objects().keys())

        for key in (self_keys - widget_keys) - {"_tag"}:
            parameter = self.param.objects()[key]
            bokeh_property = self._get_bokeh_property(parameter)
            attributes[key] = bokeh_property

        return type(class_name, bases, attributes,)

    def _get_bokeh_ts_model(self):
        pass

# See https://www.predix-ui.com/#/elements/px-app-nav
class AppNav(WebComponentWidget):
    _tag = param.String("px-app-nav", constant=True)

    items = param.String(
        '[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]'
    )

app_nav = AppNav()
app_nav_bokeh_model = app_nav._widget_type()
print(app_nav_bokeh_model)
print(app_nav_bokeh_model.items)
bryevdv commented 4 years ago

My thoughts have more been along the lines of creating a base Panel WebComponentWidget and WebComponentLayout (if possible) where the developer just needs to specify the tag, parameters and attributes to bind to.

I think we are on the same page, just looking at the problem from different angles. I am definitely coming from the lower level machinery view. I can imagine its entirely possible for there to be a convenience way of generating the adapter code, e.g. by specifying attributes on both ends, and perhaps signals in some way.

I don't know if it is possible to compile the Bokeh model once and for all. The alternative to creating a real class is creating the code string of the class.

I guess I am slightly skeptical that a single completely generic adapter could be made that would be useful for a large range of components, but I also don't have any experience with these components and would certainly love to be proved wrong!

MarcSkovMadsen commented 4 years ago

I've played a little bit around with the web components, primarely Predix.

Predix Findings

General Findings

I could not find precompiled bundles at a CDN for neither Material, Vaadin or Predict. Every guide and tutorial I find is targeted towards local installations and then the web server serving the web components as asset.

(There seems to be some for older material "components" but they are not true web components.)

I don't know how to solve this.

DISLAIMER: MAYBE I'M INCORRECT IN SOME OF THE ABOVE. I'M NOT VERY EXPERIENCED IN THESE THINGS.

image

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="utf">
    <title>Document</title>
    <script src="https://www.predix-ui.com/bower_components/webcomponentsjs/webcomponents-hi.js"></script>
    <link rel="import" href="bower_components/px-spinner/px-spinner.html" />
    <link rel="import" href="bower_components/px-app-nav/px-app-nav.html" />
    <link rel="import" href="bower_components/px-kpi/px-kpi.html" />
    <link rel="import" href="bower_components/px-heatmap-grid/px-heatmap-grid.html" />
    <link rel="import" href="bower_components/px-timeline/px-timeline.html" />
    <link rel="import" href="bower_components/px-notification/px-notification.html" />
    <style include="px-dark-theme-styles" is="custom-style"></style>
    <link rel="import" href="bower_components/px-dark-theme/px-dark-theme-styles.html" />

</head>

<body>
    <px-app-nav
        items='[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]'
        selected-route='["home"]'
        selected-meta='{"item":{"label":"Home","id":"home","icon":"px-fea:home"},"path":[{"label":"Home","id":"home","icon":"px-fea:home"}],"route":["home"],"parent":null,"children":[],"siblings":[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]}'>
    </px-app-nav>
    <br />
    <br />
    <br />
    <px-kpi spark-type="line" label="Orders" value="30M" uom="USD" status-icon="px-nav:up" status-color="green"
        status-label="12%"
        spark-data='[{"x":1397102460000,"y":0.56},{"x":1397139660000,"y":0.4},{"x":1397177400000,"y":0.43},{"x":1397228040000,"y":0.33},{"x":1397248260000,"y":0.47},{"x":1397291280000,"y":0.41},{"x":1397318100000,"y":0.26},{"x":1397342100000,"y":0.42},{"x":1397390820000,"y":0.27},{"x":1397408100000,"y":0.38},{"x":1397458800000,"y":0.36},{"x":1397522940000,"y":0.32}]'>
    </px-kpi>
    <px-heatmap-grid
        heatmap-data='[{"row":"Application 1","col":"Operator 1","value":16.639},{"row":"Application 1","col":"Operator 2","value":55.464},{"row":"Application 1","col":"Operator 3","value":91.049},{"row":"Application 1","col":"Operator 4","value":21.692},{"row":"Application 1","col":"Operator 5","value":63.163},{"row":"Application 1","col":"Operator 6","value":37.594},{"row":"Application 2","col":"Operator 1","value":61.897},{"row":"Application 2","col":"Operator 2","value":47.479},{"row":"Application 2","col":"Operator 3","value":82.82},{"row":"Application 2","col":"Operator 4","value":94.846},{"row":"Application 2","col":"Operator 5","value":6.829},{"row":"Application 2","col":"Operator 6","value":66.3},{"row":"Application 3","col":"Operator 1","value":74.379},{"row":"Application 3","col":"Operator 2","value":56.9},{"row":"Application 3","col":"Operator 3","value":44.128},{"row":"Application 3","col":"Operator 4","value":16.892},{"row":"Application 3","col":"Operator 5","value":3.908},{"row":"Application 3","col":"Operator 6","value":73.115},{"row":"Application 4","col":"Operator 1","value":61.805},{"row":"Application 4","col":"Operator 2","value":29.263},{"row":"Application 4","col":"Operator 3","value":47.576},{"row":"Application 4","col":"Operator 4","value":34.015},{"row":"Application 4","col":"Operator 5","value":29.5},{"row":"Application 4","col":"Operator 6","value":52.444},{"row":"Application 5","col":"Operator 1","value":22.53},{"row":"Application 5","col":"Operator 2","value":49.097},{"row":"Application 5","col":"Operator 3","value":29.787},{"row":"Application 5","col":"Operator 4","value":68.058},{"row":"Application 5","col":"Operator 5","value":34.698},{"row":"Application 5","col":"Operator 6","value":36.149},{"row":"Application 6","col":"Operator 1","value":98.669},{"row":"Application 6","col":"Operator 2","value":44.839},{"row":"Application 6","col":"Operator 3","value":79.367},{"row":"Application 6","col":"Operator 4","value":24.793},{"row":"Application 6","col":"Operator 5","value":9.023},{"row":"Application 6","col":"Operator 6","value":32.88},{"row":"Application 7","col":"Operator 1","value":83.108},{"row":"Application 7","col":"Operator 2","value":42.316},{"row":"Application 7","col":"Operator 3","value":66.263},{"row":"Application 7","col":"Operator 4","value":72.103},{"row":"Application 7","col":"Operator 5","value":87.811},{"row":"Application 7","col":"Operator 6","value":63.949},{"row":"Application 8","col":"Operator 1","value":15.338},{"row":"Application 8","col":"Operator 2","value":39.84},{"row":"Application 8","col":"Operator 3","value":45.119},{"row":"Application 8","col":"Operator 4","value":5.52},{"row":"Application 8","col":"Operator 5","value":5.221},{"row":"Application 8","col":"Operator 6","value":47.349},{"row":"Application 9","col":"Operator 1","value":73.392},{"row":"Application 9","col":"Operator 2","value":85.513},{"row":"Application 9","col":"Operator 3","value":67.359},{"row":"Application 9","col":"Operator 4","value":38.903},{"row":"Application 9","col":"Operator 5","value":59.048},{"row":"Application 9","col":"Operator 6","value":15.919},{"row":"Application 10","col":"Operator 1","value":75.712},{"row":"Application 10","col":"Operator 2","value":47.056},{"row":"Application 10","col":"Operator 3","value":96.009},{"row":"Application 10","col":"Operator 4","value":56.849},{"row":"Application 10","col":"Operator 5","value":10.211},{"row":"Application 10","col":"Operator 6","value":48.251}]'
        scale-min="0" scale-max="100" aggregation-type="Disabled"></px-heatmap-grid>
    <px-timeline
        timeline-data='[{"metaData":{"editedBy":"Alison Jones","editedDate":"2017-12-04 14:12:00-07","editorTitle":"Head Maintenance Technician","customIcon":"px-fea:administration"},"content":{"title":"Engine Inspection at Caledonia Shop","bodyType":"Video","body":[{"video":"https://www.youtube.com/embed/VATyEPvuQy8","headline":"Results of shop engine inspection","host":"Remote","text":"Findings attached"}]}},{"metaData":{"editedBy":"John Smith","editedDate":"2018-01-27 18:28:10-07","editorTitle":"Director of Maintenance","customIcon":"px-utl:confirmed"},"content":{"title":"A-check","bodyType":"text","body":[{"text":"A-check conducted at PredixAir maintenance hangar 3 after 260 cycles."}]}},{"metaData":{"editedBy":"PredixAir Operations","editedDate":"2018-02-12 06:37:00-07","customIcon":"px-obj:airplane"},"content":{"title":"SFO > CVG","bodyType":"LIST","body":[{"key":"Takeoff","value":"SFO - Mon Sep 12 2016 06:37:00-07"},{"key":"Arrival","value":"CVG - Mon Sep 12 2016 3:15:00-04"},{"key":"Flight Time","value":"5 hours, 43 minutes"}]}},{"metaData":{"editedBy":"GE Digital","editedDate":"2018-02-23 00:00:00-07"},"content":{"title":"Contact Details","bodyType":"table","body":[{"first":"Valentine","last":"Meyer","email":"valentinemeyer@scentric.com"},{"first":"Silva","last":"Alexander","email":"silvaalexander@gmail.com"},{"first":"Hopkins","last":"Wong","email":"hopkinswong@hotmail.com"},{"first":"Joe","last":"Sherman","email":"joejoe@yahoo.com"},{"first":"Jane","last":"Bartlett","email":"jane@scentric.com"}]}}]'
        date-format="MMM D, YYYY HH:MM" editable enhanced>
    </px-timeline>
    <px-notification type="healthy" status-icon="px-utl:filter" content="26 Filters" action-icon="px-nav:close" opened>
    </px-notification>
</body>

</html>

Install bower and components

In order to install bower the predix web components you can do something like

npm -g install bower

note where bower gets installed. Something like C:\Users\USERNAME\AppData\Roaming\npm\node_modules\bower\bin\bower on windows

In order to get a web component you should run

C:/Users/USERNAME/AppData/Roaming/npm/node_modules/bower/bin/bower install px-spinner --save
MarcSkovMadsen commented 4 years ago

I have a hypothesis that for a lot of web components I don't need to wire up a custom bokeh model.

Web component are simple html tags where user interactivity results in attribute changes. So I believe I can just use a pn.pane.Html element and react when the object parameter is changed.

Something along the lines of

import panel as pn
import param

TAG = "px-app-nav"
ITEMS = '[{"label":"Home","id":"home","icon":"px-fea:home"},{"label":"Alerts","id":"alerts","icon":"px-fea:alerts","metadata":{"openCases":"12","closedCases":"82"}},{"label":"Assets","id":"assets","icon":"px-fea:asset","children":[{"label":"Asset #1","id":"a1"},{"label":"Asset #2","id":"a2"}]},{"label":"Dashboards","id":"dashboards","icon":"px-fea:dashboard","children":[{"label":"See Live Truck View","id":"trucks","icon":"px-obj:truck"},{"label":"Track Orders","id":"orders","icon":"px-fea:orders"},{"label":"Analyze Invoices","id":"invoices","icon":"px-fea:templates"}]}]'
SELECTED_ROUTE = '["home"]'

class WebComponent(pn.pane.HTML):
    tag = param.String(TAG)
    items = param.String(ITEMS)
    selected_route = param.String(SELECTED_ROUTE)

    def __init__(self, **params):
        super().__init__(**params)

        self._set_html_tag()

    @param.depends("tag", "items", "selected_route", watch=True)
    def _update_object(self):
        self.object = f"""
        <{self.tag}
        items='{self.items}'
        selected-route='{self.selected_route}'
        </{self.tag}>
        """

    @param.depends("object", watch=True)
    def _update_parameters(self):
        NotImplementedError()
        # Extract tag, items and selected_route from object
        # Maybe only extract selected_route

Will enable me to use

image

@philippjfr . Do you think this is realistic and a good approach?

The px-app-nav can be found here https://www.predix-ui.com/#/elements/px-app-nav

MarcSkovMadsen commented 4 years ago

Ahh. Now I've learned that all "modern" web component are available via npm and compiled version are available similarly at unkpkg

See https://unpkg.com/ for an explanation of how to get a link to a precompiled bundle from npm. So far I've gotten some webcomponents working in html but the vaadin components raise an error. I will look into that.

Regarding Predix. It is sort of deprecated as it using an older version of polymer and not on npm. So I don't think that is the way to go. But at https://www.webcomponents.org/ you can find a range of web components. And

MarcSkovMadsen commented 4 years ago

I've now tried if the pn.pane.HTML can be used as a general building block. It cannot because the object is not observed. I.e. it does not trigger any events when changed.

But according to https://stackoverflow.com/questions/3103962/converting-html-string-into-dom-elements I can parse a string like <wired-radio checked>Radio Two</wired-radio> into a DOM element and then according to https://stackoverflow.com/questions/41424989/javascript-listen-for-attribute-change I can observe for mutations of the element.

Thats the next thing to explore. But that would require me to build a custom Bokeh WebComponent model.


My example (in notebook) so far is

web_component

import param
import panel as pn
pn.extension()
pn.config.sizing_mode="stretch_width"
from IPython.display import display, HTML
js = """
<script src="https://unpkg.com/@webcomponents/webcomponentsjs@2.0.0/webcomponents-loader.js"></script>
<script type="module" src="https://unpkg.com/wired-elements@0.6.4/dist/wired-elements.bundled.js"></script>
"""
display(HTML(js))
class CheckBox(pn.pane.HTML):
        tag = param.String("wired-radio", constant=True)
        js = param.List(
            ["https://unpkg.com/wired-elements@0.6.4/dist/wired-elements.bundled.js"], constant=True
        )

        attributes = param.List(["value", "checked"], constant=True)

        checked = param.Boolean(True)
        disabled = param.Boolean(False)

        def __init__(self, **params):
            super().__init__(**params)
            self._set_object()

        @param.depends("checked", watch=True)
        def _set_object(self):
            if self.checked:
                object = "<wired-radio checked>Radio Two</wired-radio>"
            else:
                object = "<wired-radio>Radio Two</wired-radio>"
            if self.object != object:
                self.object = object

        @param.depends("object", watch=True)
        def _set_checked(self):
            checked="checked" in self.object
            if self.checked != checked:
                self.checked = checked

checkbox = CheckBox()
pn.Column(pn.Param(checkbox, parameters=["object", "checked"]), checkbox)
MarcSkovMadsen commented 4 years ago

There is now a proof of concept. See https://github.com/holoviz/panel/pull/1122

web_component_poc

philippjfr commented 3 years ago

Going to close this, ReactiveHTML does let you wrap WebComponents and we may even decide to ship some wrapped components but in general I think we any full set of WebComponents should be shipped in separate library.