Question: What is the extension for?

[thoralf-gutierrez]

Hey folks!

Let me start out saying that I really appreciate bokeh! We've been using at my company very successfully for years now. Thank you all for your work.

I am curious what this extension actually does, and why folks would need it to use bokeh with JupyterHub/JupyterLab. It seems like even without this extension, I can create plots and even have bokeh apps running in notebooks.

I came to this question because we noticed that the extension was injecting bokeh 2.3.1 javascript libraries in the browser, which was clashing with the bokeh 2.4.1 that users were trying to load in their notebooks. Removing the extension fixed the problem without breaking any of our notebooks.

Thanks

[bryevdv]

Sure, some history. When Bokeh was first created in 2012 there was only the classic notebook, which allows (has always allowed) publishing JS code in output cells and having that JS code execute. That is the mechanism Bokeh used to embed its output in notebooks. It's a bit clunky, but it works, and is definitely much, much simpler for users than dealing with classic notebook extensions.

Then Jupyterlab came along. When it first came along , the Jupyter team decided that it was not safe to allow arbitrary JS execution, so they disallowed it, except via extension APIs. So out of the gate, Bokeh did not work in JupyterLab, at all. This extension was created to fix that, by embedding Bokeh output in Jupyterlab in the "official" way. In order to maintain compatibility with classic notebook, Bokeh would attempt to fall back to the original mechanism if the extension was not present.

Then at some point (I'm not sure when) the Jupyter team reversed their decision (presumably due to user complaints), and started allowing JS execution in output cells. All of the sudden (because of the fall-back), Bokeh "works" in Jupyterlab, even if the extension is not installed! I say "works" because I do think some things don't work. IIRC push_notebook and embedding Bokeh server apps in Jupyterlab require the extension nowadays.

Not sure about embedding the wrong BokehJS version, that definitely should not be the case, and I have definitely tested Bokeh 2.4 with Jupyterlab. So, I would say first double check your own package version inside Jupyterlab, in case there is e.g. some environment confusion, or different package versions than you are expecting. (cc @philippjfr @mattpap)

[thoralf-gutierrez]

Makes a ton of sense!

It sounds like the idea is that the extension would either be useful or harmless.

To give a bit more data on what we see. We are using

• JupyterHub 1.3.0
• JupyterLab 3.0.14
• jupyter-bokeh 3.0.0

To help debug on your end, when I would open my JupyterLab server, I would see:

Then, when opening a notebook with bokeh 2.4.1 installed in the kenrel, I would type

bokeh.__version__ and get 2.4.1, as expected.

BUT, the output_notebook() cell would say

while executing that cell gives the following?

I also note that the console shows I have both versions?

This issue has been really confusion because sometimes it is resolved (i.e. the output_notebook() cell shows 2.4.1) by refreshing the tab, restarting the jupyterlab server, clearing browser cache, opening in different browser, but none of these work consistently. If the stars aren't aligned, you might be trying any of these and none would work. It feels like a race condition type issue.

When I realized the extra version came from the extension, I pip uninstalled it, the 2.3.1 js didn't get loaded with JupyterLab anymore, and the notebooks were finally using the right version.

I will also note that we used to have 2.3.1 installed for notebooks (until we upgraded). I don't see how that would be relevant, but FYI in case it is. I'm not sure if the extension has 2.3.1 baked in or if it comes from something else.

Finally, we probably need to update docs here?

To use JupyterLab with Bokeh, you should at least use version 3.0 of JupyterLab. Enabling Bokeh visualizations in JupyterLab also requires the jupyter_bokeh extension to be installed.

Sounds like there is a bit more nuance that would be helpful to folks.

Thanks again for your work 🙏

[bryevdv]

I will also note that we used to have 2.3.1 installed for notebooks (until we upgraded). I don't see how that would be relevant, but FYI in case it is.

Have these notebooks never had their output cells completely cleared? If not, that would actually explain things. The old way of loading BokehJS puts all of BokehJS in an output cell, so if the notebooks have never been cleared, then it's still there. Nothing we can do about that, that's just how notebooks function.

Finally, we probably need to update docs here?

Definitely not :) It's not impossible Jupyter reverts their policy again, or makes some other change that makes the extension mandatory again in the future, in which case we'd have different versions of the docs giving mixed messages, which is always a PITA. Plus a new added burden to remember to updated them again in that event. But more importantly, full capability does require the extension. So as far as I am concerned "The extension is required, full stop" is the correct, best messaging. Simple, direct, unconditional guidance is always preferable over "X is needed, except when Y, unless Z....". At most we might want a note about clearing the notebook. (cc @tcmetzger)

[bryevdv]

Also just noting a full code grep for "2.3.1" in this repository does not turn up anything that could possibly cause BokehJS 2.3.1 specifically, to be loaded:

dev-3.9 ❯ rg "2.3.1"
yarn.lock
1463:braces@^2.3.1, braces@^2.3.2:
1749:    fsevents "~2.3.1"
1785:  version "2.3.1"
1786:  resolved "https://registry.npmjs.org/classnames/-/classnames-2.3.1.tgz"
2844:fsevents@~2.3.1:
3845:    braces "^2.3.1"
4426:  resolved "https://registry.npmjs.org/pbkdf2/-/pbkdf2-3.1.2.tgz"

package-lock.json
2984:        "fsevents": "~2.3.1"
3100:      "version": "2.3.1",
3101:      "resolved": "https://registry.npmjs.org/classnames/-/classnames-2.3.1.tgz",
4765:        "braces": "^2.3.1",
7115:      "resolved": "https://registry.npmjs.org/pbkdf2/-/pbkdf2-3.1.2.tgz",
9655:        "braces": "^2.3.1",
10050:        "braces": "^2.3.1",
12749:        "fsevents": "~2.3.1",
12849:      "version": "2.3.1",
12850:      "resolved": "https://registry.npmjs.org/classnames/-/classnames-2.3.1.tgz",
14178:            "braces": "^2.3.1",
15979:      "resolved": "https://registry.npmjs.org/pbkdf2/-/pbkdf2-3.1.2.tgz",
17996:            "braces": "^2.3.1",
18172:            "braces": "^2.3.1",

So I do think stale output cells in saved notebooks has to be the explanation.

Apart from a docs note about clearing output, it's possible the extension could also do better about actionable warning. IIRC having two version of BokehJS loaded at once used to just fall over / explode. But at some point I think support for multi-versions at once was added in the context of e.g. pages rendered by Flask or whatever. But that might cause unpredictable behavior in the context of the notebook.

[thoralf-gutierrez]

Yep, notebook cells were cleared. And I get the 2.3.1 version in the console when opening up a brand new window of JupyterHub with no notebooks open. Even after restarting the user's Jupyter server 😄

Though it definitely is a surprising coincidence that the conflicting versions match the different versions that were installed and used in notebooks on our JupyterHub. It does point to some cell in one of our old notebooks still ending up being executed somehow. Could be something going on with another extension we have installed, such as https://github.com/jpmorganchase/jupyterlab_templates 🤔 (e.g. we have notebook templates that still have the 2.3.1 js embedded in them and they get loaded somehow even though no notebooks are open?) or something completely different.

Fair enough on the docs 👍 I was thinking maybe to maybe add some nuance or have some sentences to explain what the extension actually does. I found the explanation you gave above really enlightening. But I understand the desire to keep things simple :)

Should I rename this issue to make it more clear it's about conflicting versions of bokeh being loaded in the browser when using JupyterLab?

[philippjfr]

And I get the 2.3.1 version in the console when opening up a brand new window of JupyterHub with no notebooks open. Even after restarting the user's Jupyter server.

This part is really confusing me, the extension itself does not load bokeh so there's no way bokeh should be loaded unless you open a notebook that runs bokeh.io.output_notebook. Are you sure no notebook was opened in that browser window?

[thoralf-gutierrez]

Are you sure no notebook was opened in that browser window?

Yes completely sure.

It's also possible uninstalling the extension triggered something else in JupyterLab that ended up fixing the problem, and the extension itself wasn't the issue. The uninstall could have triggered rebuilding of the JupyterLab static files? Unfortunately the system is quite complex and I don't understand the complete flow properly :/

I will report back if the issue comes back!

[bryevdv]

I was thinking maybe to maybe add some nuance or have some sentences to explain what the extension actually does.

I understand the inclination to add nuance completely, I am by default and over-explainer. But after ~15 years in OSS I definitely accept that the most accessible docs are the simplest, most direct docs. That said, a little note about what the extension is for seems like it could be useful, f you want to open an issue or PR in the main repo.

Could be something going on with another extension we have installed, such as https://github.com/jpmorganchase/jupyterlab_templates 🤔 (e.g. we have notebook templates that still have the 2.3.1 js embedded in them and they get loaded somehow even though no notebooks are open?)

I don't know enough about that other extension so say. I guess if there is some template loading BokehJS from CDN that could plausibly be related. Offhand I would say it might be worth seeing if there is any way to avoid specifying BokehJS in templates and leave it to output_notebook to do the BokehJS loading.

[bryevdv]

I am not sure there is any action to take in this repo at this point, so I will close this now.