Voila not working with hvplot/bokeh?

[ericmjl]

Issue: When serving a notebook with voila, hvplot objects are not rendered in the browser.

Here is a minimal example, using networkx and hvplot together.

import networkx as nx
import hvplot.networkx as hvnx

G = nx.erdos_renyi_graph(n=30, p=0.3)
options = [1, 2, 3]

@interact(options=options)
def plot_graph(options):
    # Ignore options, it is just a dummy to test that ipywidgets work
    # with @interact.
    return hvnx.draw(G)
    # 
    # Contrast with
    # return nx.draw(G)
    # which works.

[SylvainCorlay]

Hey @ericmjl we don't support Bokeh (yet), but we really want to :)

Libraries that are Jupyter widgets like bqplot and plotly all work as a byproduct of us supporting widgets, and we will need to do some special casing for bokeh.

[ericmjl]

Hey @SylvainCorlay thanks for looping back! Good to know it's on the team's mind. Enjoy the conference, hope the train ride is good! (saw Twitter :smile:)

[timkpaine]

Or bokeh can pivot to use ipywidgets. I suspect with the recent panel developments they probably don't want to

[maartenbreddels]

At the dashboarding workshop @philippjfr and I (mostly him) worked on wrapping bokeh widgets in an ipywidgets. I'm not sure they want to seriously support it, but it may be useful to know that people have worked on it.

[ericmjl]

ty all for chiming in! AFAICS at the moment, voila is the only simple and secure solution for serving a notebook as an app that someone else can interact with, so I'm genuinely eager to see Bokeh supported here! :smile: But no pressure, please, I trust you all have a good roadmap ahead!

[ericmjl]

Today's test with my colleague @zbarry revealed that Bokeh stuff works with voila 0.1.8 when encapsulated in a Panel container. For some reason. I think this issue might be closable then?

:smile:

[ostrokach]

@ericmjl Sorry, could you give an example of what you mean by a "panel container"?

[ericmjl]

I'm working on it right now, creating a minimal example to see if my previous comment was even correct. Be up shortly :smile:

[ericmjl]

Hmmm, I might have been overly-specific above.

Here is a minimal code example:

import panel as pn
import holoviews as hv
import pandas as pd
import numpy as np
from scipy import stats
import hvplot.pandas
hv.extension('bokeh')

# New code cell
mvn = stats.multivariate_normal(mean=[0, 0], cov=[[1, 0.8], [0.8, 1]])
mvn.rvs(1000)
df = pd.DataFrame(mvn.rvs(1000))
df.columns = ['a', 'b']
plot = df.hvplot.scatter(x='a', y='b')
plot   # this renders

# New code cell
tabs = pn.Tabs(plot)
tabs  # this renders as well

Looks like the latest release of voila does everything magically well :smile:

[ostrokach]

It still does not work for a basic interactive notebook: https://github.com/bokeh/bokeh/blob/1.3.0/examples/howto/notebook_comms/Jupyter%20Interactors.ipynb

Notebook CLI traceback:

[E 13:47:56.730 NotebookApp] Uncaught exception GET /voila/render/tmp/Jupyter%20Interactors.ipynb (::1)
    HTTPServerRequest(protocol='http', host='localhost:8889', method='GET', uri='/voila/render/tmp/Jupyter%20Interactors.ipynb', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "/home/username/miniconda3/lib/python3.7/site-packages/tornado/web.py", line 1699, in _execute
        result = await result
      File "/home/username/miniconda3/lib/python3.7/site-packages/tornado/gen.py", line 748, in run
        yielded = self.gen.send(value)
      File "/home/username/miniconda3/lib/python3.7/site-packages/voila/handler.py", line 60, in get
        result = executenb(notebook, km=km, cwd=cwd)
      File "/home/username/miniconda3/lib/python3.7/site-packages/voila/execute.py", line 133, in executenb
        return ep.preprocess(nb, resources, km=km)[0]
      File "/home/username/miniconda3/lib/python3.7/site-packages/voila/execute.py", line 88, in preprocess
        result = super(ExecutePreprocessorWithOutputWidget, self).preprocess(nb, resources=resources, km=km)
      File "/home/username/miniconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 381, in preprocess
        nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
      File "/home/username/miniconda3/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
        nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
      File "/home/username/miniconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 414, in preprocess_cell
        reply, outputs = self.run_cell(cell, cell_index)
      File "/home/username/miniconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 516, in run_cell
        self.process_message(msg, cell, cell_index)
      File "/home/username/miniconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 569, in process_message
        self.handle_comm_msg(cell.outputs, msg, cell_index)
      File "/home/username/miniconda3/lib/python3.7/site-packages/voila/execute.py", line 107, in handle_comm_msg
        state = data['state']
    KeyError: 'state'

[ericmjl]

Hmmm, perhaps I did close too early indeed.

Some other interactions weren’t going well either, as I learned. For example, Selects didn’t update my functions. Those are probably worth a separate issue, I guess?

[philippjfr]

I'll try to push out a release of ipybokeh soon as an interim solution. In the long term I'm hoping to get that merged into bokeh itself so it's available for everyone.

[thvasilo]

Is there any update on this? Due to a few of convoluted reasons I'm forced to serve Panel apps (which use Bokeh) through Voila, but while the webapp does render the plot, it loses its interactive aspects.

[philippjfr]

Working on improving it as we speak. That said you can install jupyter_bokeh now and then use pn.ipywidget to convert your HoloViews/Panel objects into an ipywidget. There's some further improvements needed to get all the functionality working, e.g. I have to merge a PR to implement appropriate throttling and some of the linked stream based interactivity in HoloViews won't work until the next release but it's all pretty much in place now and is just waiting on merging the PRs and getting new releases out.

In addition to explicit pn.ipywidgets calls, in future you will also be able to run pn.extension(comms='ipywidgets') which will render all Panel outputs as ipywidgets, which will ensure they work in Voila.

[ghost]

Hi, I'm joining the conversation because here at WeatherForce we're working on integrating React.js components in our notebooks. We've got a proof of concept working that we're planning to opensource very soon but we realized that it doesn't work in Voila, which is kind of a show stopper for us. Digging into Voila's source code it looks like it's currently very tied to ipywidgets and indeed all widgets that are known to work with Voila are either based on ipywidgets or provide a layer of compatibility with it. In fact I've just made a quick test with a very simple widget written in plain JavaScript and Python and it isn't working in Voila either.

Here is the JavaScript code:

define([
  'base/js/namespace'
], function(
  Jupyter
) {
  function load_extension() {
    console.log("*** Load extension ***");
    Jupyter.notebook.events.on('kernel_ready.Kernel', function () {
      console.log("*** Kernel ready ***");
      Jupyter.notebook.kernel.comm_manager.register_target(
        "my_comm_target", 
        function (comm, msg) {
          console.log("*** Kernel ready ***");
          comm.on_msg(function(msg){
            console.log("*** Received data from Python ***");
            const cell = Jupyter.notebook.get_msg_cell(msg.parent_header.msg_id)
            const output = cell.output_area.selector[0].getElementsByClassName('output')[0]
            output.innerHTML = `Received data from Python: ${msg.content.data}`
          })
        })
    })
  }

  return {
    load_ipython_extension: load_extension,
  };
});

And here is the Python code:

from ipykernel.comm import Comm

my_comm = Comm(target_name="my_comm_target", data={'foo': 1})
my_comm.send("Hello")

In Jupyter it displays Received data from Python: Hello as expected:

But in Voila it doesn't display anything.

Is there any plan to let Voila support widgets that aren't based on ipywidgets? If we were to submit a merge request to add support for widgets based on React.js, would you be interested to merge it in? Or should we write a compatibility layer for ipywidgets similar to what's been done for Bokeh or bqplot?

@SylvainCorlay Any thoughts on this?