search

jupyter-widgets / ipywidgets

Interactive Widgets for the Jupyter Notebook
https://ipywidgets.readthedocs.io
BSD 3-Clause "New" or "Revised" License
3.16k stars 950 forks source link

sparkmagic nbextensions not found #2058

Closed miguelarndt closed 6 years ago

miguelarndt commented 6 years ago

Hello *

Issue: jupyter-nbextension list widgetsnbextension Known nbextensions: config dir: /usr/etc/jupyter/nbconfig notebook section jupyter-js-widgets/extension enabled

My sys prefix is /usr Extensions are installed in: /usr/share/jupyter/nbextensions/jupyter-js-widgets

In detailed debug output:

jupyter-nbextension list widgetsnbextension --debug Searching ['/usr', '/root/.jupyter', '/usr/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files Looking for jupyter_config in /etc/jupyter Looking for jupyter_config in /usr/local/etc/jupyter Looking for jupyter_config in /usr/etc/jupyter Looking for jupyter_config in /root/.jupyter Looking for jupyter_config in /usr Looking for jupyter nbextension list_config in /etc/jupyter Looking for jupyter nbextension list_config in /usr/local/etc/jupyter Looking for jupyter nbextension list_config in /usr/etc/jupyter Looking for jupyter nbextension list_config in /root/.jupyter Looking for jupyter nbextension list_config in /usr Known nbextensions: Paths used for configuration of common: /root/.jupyter/nbconfig/common.json Paths used for configuration of notebook: /root/.jupyter/nbconfig/notebook.json Paths used for configuration of tree: /root/.jupyter/nbconfig/tree.json Paths used for configuration of edit: /root/.jupyter/nbconfig/edit.json Paths used for configuration of terminal: /root/.jupyter/nbconfig/terminal.json Paths used for configuration of common: /usr/etc/jupyter/nbconfig/common.json Paths used for configuration of notebook: /usr/etc/jupyter/nbconfig/notebook.d/widgetsnbextension.json /usr/etc/jupyter/nbconfig/notebook.json config dir: /usr/etc/jupyter/nbconfig notebook section widgetsnbextension enabled

Any hints on how to get widgets working (for actual sparkmagic) are more than welcome, since i have no clue why the error message tells the extensions are missing

TIA Micha

jasongrout commented 6 years ago

It looks like things should work, based on what you pasted above. To get rid of the error, you just need to delete the mention of widgetsnbextension in your config, which according to the log above, looks like it is in /usr/etc/jupyter/nbconfig

Here's what your output should look like:

% jupyter nbextension list
Known nbextensions:
  config dir: /[snip]/etc/jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK

Longer backstory: A long long time ago, the widgets nbextension was installed under the jupyter-js-widgets name. A long time ago, we split off the nbextension to a new python package, called widgetsnbextension, and changed the nbextension install directory to widgetsnbextension. This caused backwards compatibility problems with people that had older versions installed in the jupyter-js-widgets directory (it wouldn't upgrade the extension), so we switched back to having the python widgetsnbextension install its nbextension into the jupyter-js-widgets nbextension directory. So now, for a while (since before 7.0, IIRC), the python widgetsnbextension package installs its nbextension into the jupyter-js-widgets nbextension directory.

It looks like you have a leftover widgetsnbextension nbextension enabled from some old install. The appropriate thing to do is to just delete the line in your config enabling that extension so it won't go looking for it and printing an error.

jasongrout commented 6 years ago

(This is all assuming that you are running ipywidgets 7.0)

miguelarndt commented 6 years ago

Jason,

thx, help more than appreciated. Keeping to ipywidgets 7.0 i can do.

Currently it looks like that:

pwd /usr/etc/jupyter/nbconfig

[root@atgrzsl2653 nbconfig]# cat notebook.json { "CodeCell": { "cm_config": { "indentUnit": 2 } }, "load_extensions": { "widgetsnbextension": true, "jupyter-js-widgets/extension": true } }

If i undestand your hint correct, i change to:

{ "CodeCell": { "cm_config": { "indentUnit": 2 } }, "load_extensions": { "jupyter-js-widgets/extension": true } }

and now indeed get:

jupyter nbextension list Known nbextensions: config dir: /usr/etc/jupyter/nbconfig notebook section jupyter-js-widgets/extension enabled

So issue reported resolved, as it looks to me.

I am curious and will test immediatly, whether now a widget pops up, when making a connection in sparkmagic :-), thats my final goal

If so, you have more than made my day !!!

thx

Micha

Am 2018-04-26 16:20, schrieb Jason Grout:

It looks like things should work, based on what you pasted above. To get rid of the error, you just need to delete the mention of widgetsnbextension in your config, which according to the log above, looks like it is in /usr/etc/jupyter/nbconfig

Here's what your output should look like:

% jupyter nbextension list Known nbextensions: config dir: /[snip]/etc/jupyter/nbconfig notebook section jupyter-js-widgets/extension enabled

  • Validating: OK

Longer backstory: A long long time ago, the widgets nbextension was installed under the jupyter-js-widgets name. A long time ago, we split off the nbextension to a new python package, called widgetsnbextension, and changed the nbextension install directory to widgetsnbextension. This caused backwards compatibility problems with people that had older versions installed in the jupyter-js-widgets directory (it wouldn't upgrade the extension), so we switched back to having the python widgetsnbextension install its nbextension into the jupyter-js-widgets nbextension directory. So now, for a while (since before 7.0, IIRC), the python widgetsnbextension package installs its nbextension into the jupyter-js-widgets nbextension directory.

It looks like you have a leftover widgetsnbextension nbextension enabled from some old install. The appropriate thing to do is to just delete the line in your config enabling that extension so it won't go looking for it and printing an error.

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384658300 [2] https://github.com/notifications/unsubscribe-auth/Air_NDmxH4jyMM0suNbzzTGJOmQFk-rpks5tsdeegaJpZM4TlFfz

miguelarndt commented 6 years ago

with my thanks, i am semi-happy now :-)

I assume this is not in the scope of ipywidgets anymore, so i describe just in case

and thank for your perfect help before.

If i open in sparkmagic a spark kernel and do:

%reload_ext sparkmagic.magics %manage_spark

I get the ascii text output: "A Jupyter Widget"

But no real widget popping up

thx

Micha

Am 2018-04-26 16:20, schrieb Jason Grout:

(This is all assuming that you are running ipywidgets 7.0)

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384658388 [2] https://github.com/notifications/unsubscribe-auth/Air_NBp4af-m6pIRUKEmLmCRaL_m1u5fks5tsdergaJpZM4TlFfz

jasongrout commented 6 years ago

I get the ascii text output: "A Jupyter Widget"

Are there any javascript console errors in your browser?

miguelarndt commented 6 years ago

In Firefox Debug Console:

Error: Object 'jupyter.widget' not found in registry main.d5f055b043c45d52f90d.js:1:1461258

and

Error: Invalid response: 502 Bad Gateway Stack trace: t@https://abcd1234/user/myuser/lab/static/main.d5f055b043c45d52f90d.js:1:18004 .CQtt/p</e.prototype.get/<@https://abcd1234/user/myuser/lab/static/main.d5f055b043c45d52f90d.js:1:521441

Am 2018-04-26 17:29, schrieb Jason Grout:

I get the ascii text output: "A Jupyter Widget"

Are there any javascript console errors in your browser?

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384684224 [2] https://github.com/notifications/unsubscribe-auth/Air_NErAU3yplHIsOZN6d6mSFofczzevks5tsefWgaJpZM4TlFfz

jasongrout commented 6 years ago

It seems that the widgetsnbextension js is not getting loaded in your browser. Can you check to see that it is? In the "Debugger" inspector tab in FF, or the "Sources" tab in Chrome, you should see the nbextensions/jupyter-js-widgets/extension.js file loaded (screenshot below in Firefox):

screen shot 2018-04-26 at 8 54 20 am
miguelarndt commented 6 years ago

Looks like widgetsnbextensions js is not loaded by my Firefox ( RHEL 6 Standard Firefox as Client)

\m

Am 2018-04-26 17:55, schrieb Jason Grout:

It seems that the widgetsnbextension js is not getting loaded in your browser. Can you check to see that it is? In the "Debugger" inspector tab in FF, or the "Sources" tab in Chrome, you should see the nbextension/jupyter-js-widgets/extension.js file loaded (screenshot below in Firefox):

[1]

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [2], or mute the thread [3].

Links:

[1] https://user-images.githubusercontent.com/192614/39317089-99fb45ea-492f-11e8-82df-c335ec628d8d.png [2] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384693983 [3] https://github.com/notifications/unsubscribe-auth/Air_NAQHBt1bpKeup_nTwXBpCt4S8gPdks5tse4CgaJpZM4TlFfz

jasongrout commented 6 years ago

Is there an error in requesting it (in the networks panel)? Or is there an error in the notebook server log showing that it was requested?

miguelarndt commented 6 years ago

i do not see an error / request in the jupyterhub log, if that is meant by notebook server log

I will yet check for networks panel errors

Am 2018-04-26 19:00, schrieb Jason Grout:

Is there an error in requesting it (in the networks panel)? Or is there an error in the notebook server log showing that it was requested?

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384715088 [2] https://github.com/notifications/unsubscribe-auth/Air_NMjDWHqewTdHzVokGjs39VqkNdIAks5tsf1HgaJpZM4TlFfz

miguelarndt commented 6 years ago

Jason,

just two questions, in order to not block your timeslots:

-is there any way i can debug why the firefox is not aware of widgetsnbextension ?

Can i make the extension known "manually" to the browser, or can i execute python testcode that calls the widget ?

FYI, it is RHEL 7.4 and the standard "out of the box" firefox of RHEL 7.4, al Jupyterhub / Lab and sparkmagic components installed with pip, python 3.4, not used anaconda

i excuse about putting so much questions, completely accept in case you can not take time to answer !!

Micha

Is there an error in requesting it (in the networks panel)? Or is there an error in the notebook server log showing that it was requested?

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384715088 [2] https://github.com/notifications/unsubscribe-auth/Air_NMjDWHqewTdHzVokGjs39VqkNdIAks5tsf1HgaJpZM4TlFfz

jasongrout commented 6 years ago

Sorry, I was waiting to hear your answer about the networks panel errors, since you said you were going to check that. What we're trying to do here is to see if the notebook is trying to load the right javascript file. That will tell us if the error is in the notebook not knowing to load the file, or if the error is in the file not being there.

jasongrout commented 6 years ago

(your jupyter nbextension list says the nbextension is enabled, so from that, it should be trying to get the right javascript. If you don't see an error from it trying to load the javascript, then that's puzzling...)

miguelarndt commented 6 years ago

Jason

I will dedicate an extended debug session next monday / tuesday and hopefully provide answers to your hints and questions. Thanks for your explan, now i understand completely what i should go hunting for

Am 27.04.2018 um 14:07 schrieb Jason Grout notifications@github.com:

(your jupyter nbextension list says the nbextension is enabled, so from that, it should be trying to get the right javascript. If you don't see an error from it trying to load the javascript, then that's puzzling...)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jasongrout commented 6 years ago

If it is trying to load the javascript, but having a 404 error and just not finding it, the next thing I would do is run the notebook in debug mode jupyter notebook --debug, which lists for each file that is requested the exact location on disk that it expects to find it (the log lines starting with Path). That helps us track down exactly where it expects to find the widget javascript.

miguelarndt commented 6 years ago

Jason,

this is the error message i get with ipywidgets 7.2.1 and widgetsnbextension 3.2.1

MagicsControllerWidget(children=(Tab(children=(ManageSessionWidget(children=(HTML(value='
'), HTML(value='\u2026

with ipywidgets 7.0.0 and widgetsnbextension-3.0.8 the same action results in:

A Jupyter Widget

i started jupyterhub with --debug flag, no failing "GET" calls

all calls similar to this one:

I 2018-04-30 16:46:16.254 SingleUserNotebookApp log:122] 200 GET /user/myuser/api/contents/?content=1&1525099576235 (myuser@127.0.0.1) 14.80ms

I accessed directly with browser on localhost, in order to avoid causing issues with the reverse proxy.

jupyter nbextension list Known nbextensions: config dir: /usr/etc/jupyter/nbconfig notebook section jupyter-js-widgets/extension enabled

best

Micha

Am 2018-04-27 14:45, schrieb Jason Grout:

If it is trying to load the javascript, but just not finding it, the next thing I would do is run the notebook in debug mode jupyter notebook --debug, which lists for each file that is requested the exact location on disk that it expects to find it (the log lines starting with Path). That helps us track down exactly where it expects to find the widget javascript.

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-384959398 [2] https://github.com/notifications/unsubscribe-auth/Air_NO7BvW_DPN_yHwwAKRhN6vlfjNOdks5tsxLYgaJpZM4TlFfz

jasongrout commented 6 years ago

Are there any browser network requests for something that looks like nbextensions/jupyter-js-widgets? Are there any requests in the Server log that look like that?

It seems that the widgets javascript is not loaded in your browser. The next thing is to figure out if the browser is trying to load it.

jasongrout commented 6 years ago

Are you using Chrome or Firefox? If Firefox, here is the browser tool I'm talking about: https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor. Make sure to reload the notebook page to see all of the network requests.

miguelarndt commented 6 years ago

Jason,

thx for bearing with me and your infinite patience. Yes, i think i opened the right network monitor in firefox expert menu, but found no reference to nbextensions loading or being missed in filesystem.

To verify, i do not trample on your time by my stupidity i did an complete uninstall of 

* jupyterhub / jupyterlab
* some modules involved, especially nbextensions / ipywidgets
* then a complete reinstall without jupyterlab (!)

Diagnosis: Now i get the connection widget after calling %manage_spark

Is it possible, that there is a conflict with jupyterlab and nbextensions / ipywidgets ?

For now i will tell the researchers to live with jupyterhub without jupyterlab, since their focus is sparkmagic and talking to the livy server.

To you more than a huge thank you, your readiness to help and give precise information is more than exceptional !

Micha

Are you using Chrome or Firefox? If Firefox, here is the browser tool I'm talking about: https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor. Make sure to reload the page to see all of the network requests.

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-385428125 [2] https://github.com/notifications/unsubscribe-auth/Air_NC6s9EQcJlrKlugfQTJJf_LQb89oks5ttykXgaJpZM4TlFfz

jasongrout commented 6 years ago

Wait, were you using JupyterLab, not the classic Jupyter Notebook? I was assuming in all of the above we were in the classic Jupyter notebook, not JupyterLab.

jasongrout commented 6 years ago

To enable widgets in JupyterLab, you need to install a JupyterLab extension:

jupyter labextension install @jupyter-widgets/jupyterlab-manager
jasongrout commented 6 years ago

See http://ipywidgets.readthedocs.io/en/latest/user_install.html#installing-the-jupyterlab-extension

jasongrout commented 6 years ago

Somehow I missed the mention of lab path in one of your debug postings, and concentrated on your use of widgetsnbextension as indicating we were dealing with the classic notebook. JupyterLab has its own extension system, and doesn't use the nbextension system from the classic notebook. The widgetsnbextension package is for the classic notebook. The extension listed above in the ipywidgets JupyterLab installation instructions is for JupyterLab.

I think your problem may be solved as easily as just:

jupyter labextension install @jupyter-widgets/jupyterlab-manager

You'll need nodejs installed in order to build JupyterLab with this extension.

miguelarndt commented 6 years ago

Jason,

thx & absolutely my fault. Only in my last mail i mentioned jupyterlab, since i thought it to be irrelevant for the issue.

Jason, your solution works perfectly well for me :-) with jupyterlab ! !!!!

And thus verifies, that one should be more carefully than me about considering s.t. "irrelevant"

Is it possible to hint a short explanation, when/why a notebook extension is one time installed via pip ( or conda equivalent) and here via jupyter labextension install ?

Is that just caused by the subsystem managing install dependencies or is there a more complex reason behind ?

you made my day with your help definitely !

Micha

Am 2018-04-30 18:25, schrieb Jason Grout:

Somehow I missed the mention of lab path in one of your debug postings, and concentrated on your use of widgetsnbextension as indicating we were dealing with the classic notebook. JupyterLab has its own extension system, and doesn't use the nbextension system from the classic notebook. The widgetsnbextension package is for the classic notebook. The extension listed above in the ipywidgets JupyterLab installation instructions is for JupyterLab.

I think your problem may be solved as easily as just:

jupyter labextension install @jupyter-widgets/jupyterlab-manager

You'll need nodejs installed in order to build JupyterLab with this extension.

-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

Links:

[1] https://github.com/jupyter-widgets/ipywidgets/issues/2058#issuecomment-385451701 [2] https://github.com/notifications/unsubscribe-auth/Air_NBeyXbRCFSYLIrKaWcvwJGtWNUoTks5ttzsNgaJpZM4TlFfz

jasongrout commented 6 years ago

Is it possible to hint a short explanation, when/why a notebook extension is one time installed via pip ( or conda equivalent) and here via jupyter labextension install ?

The classic notebook and JupyterLab have entirely different extension mechanisms. In JupyterLab, an extension is a javascript package (npm package) that is built into a new build of the JupyterLab application. In the classic notebook, the extension system works with the python packaging system and that is the only way to distribute your javascript code.

jasongrout commented 6 years ago

Closing as resolved (I'm glad it's working now!). Feel free to comment further if you have other questions, though.

SahithiGunna commented 6 years ago

To enable widgets in JupyterLab, you need to install a JupyterLab extension:

jupyter labextension install @jupyter-widgets/jupyterlab-manager

I am running into this issue when installing jupyterlab-manager. Can you please provide some insights here. I am using jupyterlab here.

[root@bluedata-375 site-packages]# jupyter labextension install @jupyter-widgets/jupyterlab-manager

/opt/anaconda3/bin/npm pack @jupyter-widgets/jupyterlab-manager jupyter-widgets-jupyterlab-manager-0.38.1.tgz

Errored, use --debug for full output: ValueError: "@jupyter-widgets/jupyterlab-manager@0.38.1" is not compatible with the current JupyterLab Conflicting Dependencies: JupyterLab Extension Package

=0.15.4-0 <0.16.0-0 >=0.19.1-0 <0.20.0-0 @jupyterlab/application =1.0.6-0 <2.0.0-0 >=2.0.0-0 <3.0.0-0 @jupyterlab/coreutils =0.15.4-0 <0.16.0-0 >=0.19.1-0 <0.20.0-0 @jupyterlab/notebook =0.15.4-0 <0.16.0-0 >=0.19.1-0 <0.20.0-0 @jupyterlab/rendermime =1.1.4-0 <2.0.0-0 >=3.0.0-0 <4.0.0-0 @jupyterlab/services

Also, I am trying to install jupyterlab-manager due to this issue. screen shot 2018-11-07 at 6 03 49 pm

Thanks, Sahithi

vidartf commented 6 years ago

I would recommend updating jupyterlab. If that is not possible, see the readme of the jupyterlab-manager package for a comparability table. Note: newer versions of lab will automatically pick the right version, so this should only be a needed step for older versions of lab.

SahithiGunna commented 6 years ago

I see. Let me try that.Thanks, @vidartf