External resources belonging to custom models fail to load inside of Jupyter/IPython notebooks

[patricktokeeffe]

The LatexLabel example is a great resource but I'm observing problems using it with output_notebook(): plots are rendered with essentially no height until the browser window is resized. Removing the LatexLabel is all it takes to restore normality. This problem does not appear for output_file().

Here is the output produced by show()...

...and it's 'fixed' as soon as I change the window dimensions slightly.

Using bokeh 0.12.2 with Anaconda 4.1.0 (Py35) on Win7 x64 + Chromium 53.0

xref #647

[bryevdv]

ping @birdsarah @canavandl @philippjfr in case anyone has any quick ideas about the notebook specifically.

@patricktokeeffe it' looks like you are just using LaTex for an arrow? If that's the case then I'd suggest the Arrow Bokeh annotation as a potential immediate workaround

[patricktokeeffe]

@bryevdv I used LaTeX to display some relevant formulas below the plot. For my case, the Div widget could potentially work, but it doesn't properly escape the necessary <script> tags.

I also spoke too soon.. resizing the browser restores the plot dimensions, but the label is missing and so are titles and axes labels:

versus:

[canavandl]

The problem appears to be that how Bokeh is loading the external resources (katex.min.js and katex.min.css from the CDN) fails to add the resources to the global namespace. The plot rendering fails then when the LaTexLabel implementation hits a line where it expects katex to be present in that global namespace and results results in the lack of output you're seeing.

A better solution would be to have the custom model compilation step bundle the external resource so that it it be imported in the module via require.

[canavandl]

@patricktokeeffe - since this is a general problem affecting all custom models that use external resources in the notebook, i'm going to rename your issue to be a bit more specific to the bug.

[canavandl]

ping @mattpap

[mattpap]

After recent changes we have a different problem to solve first:

TypeError: <module '__main__' (built-in)> is a built-in class

This problem is not inherently connected to notebook. Whenever custom models are used in a module without __file__ attribute, they will fail like this. E.g. it's sufficient to try to debug custom models with ipdb, to get the same result. I had to fall back to pdb to proceed with debugging (apparently there is at least one area where pdb is better than ipdb).

[mattpap]

The original problem is with define() (from IPython's module loader) being picked up by katex, which now assumes it's in IPython's module context and uses define() to create a module for itself. However, it should be registering itself on window. I'm not sure if IPython could behave better here. katex uses the standard approach for module loader detection, so it's not at fault here. We could, however, try to protect libraries we load from external conditions (like foreign module loaders laying around). We do this already in bokeh(.min).js bundle, but for now I wasn't able to achieve the same in autoload code.

[mattpap]

Just note that we try to figure out how to protect libraries anyway, because if it's not IPython, it will be another hosting environment that does that.

[mattpap]

Given that external resources are loaded through createElement("script"), which makes them evaluate in the global context, not the context of autoloader, then we may need to fetch scripts like json files and eval() them instead, but this approach is considered unsafe (XSS).

[mattpap]

@damianavila, maybe you have some feedback regarding IPython exposing define()?

[damianavila]

Probably @blink1073 and @gnestor have some better insights on this one...

[blink1073]

I was only aware of a single window.define in the classic notebook, which is a pre-configured requirejs loader, I don't know what is meant by the IPython define().

[mattpap]

I don't know what is meant by the IPython define()

@blink1073, window.define.

[blink1073]

Ah, existing notebook extensions rely on window.define, so I don't see that API going away. We have explicitly avoiding having window.define or window.require in JupyterLab from the start.

[bryevdv]

@blink1073 do you have any guidance or suggestions?

[blink1073]

I think the only true solution is to use UMD as well when Bokeh is embedded in another page, since you don't know whether an AMD loader will be present.

[mattpap]

The simplest solution would be to load external dependencies with XMLHttpRequest (and wrap fetched code in (function() { var define = undefined; var module = undefined; ... })();), but that will fail due to cross-domain policy. Apparently you can load scripts using createElement("script") without problems, but using XMLHttpRequest for the same thing is the source of all evil. Maybe someone can explain the difference in security of both approaches, because I don' get it? Maybe I miss something obvious.

[blink1073]

AFAIK, creating a script tag and setting itssrc is functionally equivalent to eval() on the same source fetched using XMLHttpRequest, since they are both evaluated in the global context.

[Karel-van-de-Plassche]

Running the ion.rangeSlider example also does not work within a Jupyter notebook in bokeh 0.12.4. First run of output_file() does nothing, second run fails with:

Javascript error adding output!
TypeError: jQuery(...).ionRangeSlider is not a function
See your browser Javascript console for more details.

TypeError: jQuery(...).ionRangeSlider is not a function[Learn More]  main.min.js:232:24
    inline_js</<["custom/ionrangeslider.ion_range_slider"]</exports.IonRangeSliderView</IonRangeSliderView.prototype.render http://karel-work.ele.tue.nl:8888/static/notebook/js/main.min.js:232:24
    Bokeh</<["models/layouts/layout_dom"]</r.LayoutDOMView</e.prototype.bind_bokeh_events/</< https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:16:31398
    Bokeh</<["core/events"]</u/t[e]< https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:7:2093
    Bokeh</<.underscore</</x.before/< https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:140:9267
    Bokeh</<["core/events"]</p https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:7:2534
    Bokeh</<["core/events"]</c https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:7:2399
    Bokeh</<["core/events"]</i https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:7:297
    Bokeh</<["core/events"]</r.Events.trigger https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:7:2292
    Bokeh</<.document</r.Document</t.prototype._resize https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:9:5010
    Bokeh</<.document</r.Document</t.prototype.resize https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:9:4379
    bound  self-hosted
    Bokeh</<.underscore</</x.delay/< https://cdn.pydata.org/bokeh/release/bokeh-0.12.4.min.js:140:8190

[philippjfr]

I think the way forward on this issue is to add a branch specifically for the classic notebook which can exploit the fact that require is available and load the scripts that way. The main issue then is that we then need to find a way to map the JS resources to specific module names that can be exposed on the window. That could be done by allowing dictionary mappings on __javascript__ to give each script a module name.

When I just hard-coded it, this approach seemed to work well. In this example I was loading a Plotly bokeh model:

require.config({paths: {'Plotly': 'https://cdn.plot.ly/plotly-latest.min'}})
require(['Plotly'], function(Plotly) { window.Plotly = Plotly; on_load(); })

[philippjfr]

Alternatively how hacky is it to simply temporarily set window.define = undefined while the external scripts are added, because that's a 4-line fix.

[philippjfr]

Setting window.define = undefined is definitely not a great thing to so it would certainly be best to use require properly. That said it's certainly not straightforward since we somehow need to make the module names available and in certain cases would even have to define requireJS shims.

@mattpap Do you have any suggestions? Do you see an easy way to provide the module names to autoload_js or can you see a different approach at this point? Is the hacky approach of temporarily unsetting window.define a reasonable workaround since it currently does not work at all and that would make it work in many if not all cases?

[bryevdv]

@philippjfr @mattpap do we know if this issue is still relevant?