Error executing Jupyter command 'kernelspec'

[drom]

I am getting Error executing Jupyter command 'kernelspec': error.

Here are the steps to reproduce:

drom@drom:~> ijs --version                                                            
5.0.20                                                                                

nvm i 9
Downloading and installing node v9.2.0...
Now using node v9.2.0 (npm v5.5.1)

drom@drom:~> npm i -g ijavascript
/home/drom/.nvm/versions/node/v9.2.0/bin/ijs -> /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/bin/ijavascript.js
/home/drom/.nvm/versions/node/v9.2.0/bin/ijsconsole -> /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/bin/ijsconsole.js
/home/drom/.nvm/versions/node/v9.2.0/bin/ijsinstall -> /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/bin/ijsinstall.js
/home/drom/.nvm/versions/node/v9.2.0/bin/ijskernel -> /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/lib/kernel.js
/home/drom/.nvm/versions/node/v9.2.0/bin/ijsnotebook -> /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/bin/ijsnotebook.js

> zeromq@4.6.0 install /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/node_modules/zeromq
> node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)

prebuild-install info begin Prebuild-install version 2.4.0
prebuild-install info looking for local prebuild @ prebuilds/zeromq-v4.6.0-node-v59-linux-x64.tar.gz
prebuild-install info looking for cached prebuild @ /home/drom/.npm/_prebuilds/https-github.com-zeromq-zeromq.js-releases-download-v4.6.0-zeromq-v4.6.0-node-v59-linux-x64.tar.gz
prebuild-install info found cached prebuild 
prebuild-install info unpacking @ /home/drom/.npm/_prebuilds/https-github.com-zeromq-zeromq.js-releases-download-v4.6.0-zeromq-v4.6.0-node-v59-linux-x64.tar.gz
prebuild-install info unpack resolved to /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/node_modules/zeromq/build/Release/zmq.node
prebuild-install info unpack required /home/drom/.nvm/versions/node/v9.2.0/lib/node_modules/ijavascript/node_modules/zeromq/build/Release/zmq.node successfully
prebuild-install info install Successfully installed prebuilt binary!                 

+ ijavascript@5.0.20                                                                  
added 58 packages in 3.488s                                                           

drom@drom:~> ijsinstall --install=local
Error running `jupyter kernelspec install --replace /tmp/a9bfda20c0c77d72a2495bc42b52bd93/javascript --user`                                                                        
Error: Command failed: jupyter kernelspec install --replace /tmp/a9bfda20c0c77d72a2495bc42b52bd93/javascript --user                                                                 
Error executing Jupyter command 'kernelspec': [Errno 2] No such file or directory     

Error executing Jupyter command 'kernelspec': [Errno 2] No such file or directory     

[n-riesco]

How did you install jupyter? It looks like your installation is missing jupyter-kernelspec.

[drom]

I am on OpenSuSe Tumbleweed and installed via Yast

drom@drom:~> jupyter --version                                                                
4.3.0

[n-riesco]

Please, make sure python-jupyter_client is also installed. I've just checked OpenSuse's repository and their python-jupyter package doesn't list python-jupyter_client as a dependence.

[drom]

I see python3-jupyter_client installed. python2-jupyter_client is not. Do I need it?

i+ | python3-jupyter                    | Environment for interactive computing                           | package   
i  | python3-jupyter_client             | Jupyter protocol implementation and client libraries            | package   
i  | python3-jupyter_console            | Jupyter terminal console                                        | package   
i  | python3-jupyter_core               | Base package on which Jupyter projects rely                     | package   
i  | python3-jupyter_ipykernel          | IPython Kernel for Jupyter                                      | package   
i  | python3-jupyter_ipyparallel        | Interactive Parallel Computing with IPython                     | package   
i  | python3-jupyter_ipython            | Rich architecture for interactive computing with Python         | package   
   | python3-jupyter_ipython-iptest     | Tools for testing packages that rely in python3-jupyter_ipython | package   
i+ | python3-jupyter_ipywidgets         | IPython HTML widgets for Jupyter                                | package   
i  | python3-jupyter_nbconvert          | Converting Jupyter Notebooks                                    | package   
   | python3-jupyter_nbconvert-latex    | LaTeX support for nbconvert                                     | package   
i  | python3-jupyter_nbformat           | The Jupyter Notebook format                                     | package   
   | python3-jupyter_nbsphinx           | Jupyter Notebook Tools for Sphinx                               | package   
i+ | python3-jupyter_notebook           | Jupyter Notebook                                                | package   
   | python3-jupyter_notebook-latex     | LaTeX support for the Jupyter Notebook                          | package   
i  | python3-jupyter_qtconsole          | Jupyter Qt console                                              | package   
   | python3-jupyter_sphinx_theme       | Jupyter Sphinx Theme                                            | package   
i+ | python3-jupyter_widgetsnbextension | IPython HTML widgets for Jupyter                                | package   

[drom]

I have installed python2-jupyter* packages. The same error.

[n-riesco]

Could you output what you get when you run which jupyter and which jupyter-kernelspec?

[drom]

drom@drom:~> which jupyter
/usr/bin/jupyter

I don't have jupyter-kernelspec

[n-riesco]

Please, could you run rpm -ql python3-jupyter_client to list the files installed by python3-jupyter_client, and check whether jupyter-kernelspec is there?

[drom]

drom@drom:~> rpm -ql python3-jupyter_client | grep kernelspec
/usr/lib/python3.6/site-packages/jupyter_client/__pycache__/kernelspec.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/jupyter_client/__pycache__/kernelspec.cpython-36.pyc
/usr/lib/python3.6/site-packages/jupyter_client/__pycache__/kernelspecapp.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/jupyter_client/__pycache__/kernelspecapp.cpython-36.pyc
/usr/lib/python3.6/site-packages/jupyter_client/kernelspec.py
/usr/lib/python3.6/site-packages/jupyter_client/kernelspecapp.py
/usr/lib/python3.6/site-packages/jupyter_client/tests/__pycache__/test_kernelspec.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/jupyter_client/tests/__pycache__/test_kernelspec.cpython-36.pyc
/usr/lib/python3.6/site-packages/jupyter_client/tests/test_kernelspec.py

[n-riesco]

I think that explains the issue. I'm using Ubuntu, and this is what I get:

$ dpkg-query -L jupyter-client 
/.
/usr
/usr/bin
/usr/bin/jupyter-kernelspec
/usr/share
/usr/share/doc
/usr/share/doc/jupyter-client
/usr/share/doc/jupyter-client/copyright
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/jupyter-kernelspec.1.gz
/usr/share/doc/jupyter-client/changelog.Debian.gz

$ cat /usr/bin/jupyter-kernelspec 
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'jupyter-client==4.4.0','console_scripts','jupyter-kernelspec'
__requires__ = 'jupyter-client==4.4.0'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('jupyter-client==4.4.0', 'console_scripts', 'jupyter-kernelspec')()
    )

I've just checked https://software.opensuse.org/package/python3-jupyter_client and it looks like Tumbleweed already provides version 5.1.0 and includes some fixes. Would you be able to upgrade and check if this fixes the missing jupyter-kernelspec?

[drom]

Yes, I have latest python3-jupyter_client version 5.1.0 from the very beginning. Even so python3-jupyter_core version 4.3.0

drom@drom:~> jupyter --version
4.3.0

[n-riesco]

I'm not sure what to suggest next:

• easiest option: install jupyter using pip as described in http://jupyter.org/install.html
• hacker's option: create your own /usr/bin/jupyter-kernelspec similar to Ubuntu's example:

  
  $ cat /usr/bin/jupyter-kernelspec 

!/usr/bin/python3

EASY-INSTALL-ENTRY-SCRIPT: 'jupyter-client==4.4.0','console_scripts','jupyter-kernelspec'

requires = 'jupyter-client==4.4.0' import re import sys from pkg_resources import load_entry_point

if name == 'main': sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('jupyter-client==4.4.0', 'console_scripts', 'jupyter-kernelspec')() )

[drom]

Thank you for your help. I have tried second method with version modifications:

#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'jupyter-client==5.1.0','console_scripts','jupyter-kernelspec'
__requires__ = 'jupyter-client==5.1.0'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('jupyter-client==5.1.0', 'console_scripts', 'jupyter-kernelspec')()
    )

But still getting an error:

drom@localhost:~> ijsinstall --install=local
Error running `jupyter kernelspec install --replace /tmp/6e25f7e8f5cd3186da1aced868731eb8/javascript --user`
Error: Command failed: jupyter kernelspec install --replace /tmp/6e25f7e8f5cd3186da1aced868731eb8/javascript --user
Traceback (most recent call last):
  File "/usr/bin/jupyter-kernelspec", line 11, in <module>
    load_entry_point('jupyter-client==5.1.0', 'console_scripts', 'jupyter-kernelspec')()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2751, in load_entry_point
    raise ImportError("Entry point %r not found" % ((group, name),))
ImportError: Entry point ('console_scripts', 'jupyter-kernelspec') not found

Traceback (most recent call last):
  File "/usr/bin/jupyter-kernelspec", line 11, in <module>
    load_entry_point('jupyter-client==5.1.0', 'console_scripts', 'jupyter-kernelspec')()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2751, in load_entry_point
    raise ImportError("Entry point %r not found" % ((group, name),))
ImportError: Entry point ('console_scripts', 'jupyter-kernelspec') not found

[n-riesco]

I'll try to install OpenSuse in a VM and see if I can figure out a workaround.

[n-riesco]

I haven't tested this on OpenSuse, but I've adapted the script to work around the missing entry point. This is working in Ubuntu, could you check if it works in OpenSuse?

• Create /usr/bin/jupyter-kernelspec

  
  $ cat /usr/bin/jupyter-kernelspec
  #!/usr/bin/python3

import re import sys

from jupyter_client.kernelspecapp import KernelSpecApp

if name == 'main': sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0]) sys.exit( KernelSpecApp.launch_instance() )

- ensure it has execution rights

$ chmod +x /usr/bin/jupyter-kernelspec

- check it works

$ /usr/bin/jupyter-kernelspec list Available kernels: javascript /home/user/.local/share/jupyter/kernels/javascript python3 /usr/share/jupyter/kernels/python3

[drom]

Yes, you code give me the following result:

drom@drom:~> jupyter-kernelspec list
Available kernels:
  python3       /usr/lib/python3.6/site-packages/ipykernel/resources
  python_2.7    /usr/share/jupyter/kernels/python_2.7
  python_3.6    /usr/share/jupyter/kernels/python_3.6

[drom]

And install works now!

drom@drom:~> ijsinstall --install=local
drom@drom:~> jupyter-kernelspec list
Available kernels:
  python3       /usr/lib/python3.6/site-packages/ipykernel/resources
  javascript    /home/drom/.local/share/jupyter/kernels/javascript
  python_2.7    /usr/share/jupyter/kernels/python_2.7
  python_3.6    /usr/share/jupyter/kernels/python_3.6

[n-riesco]

ijsinstall should work now Ah I see it's worked :)

[drom]

[drom]

Awesome! It works now. Thank you very much for your help.

[drom]

I have 4 more OpenSuSe Tumbleweed machines to go. I am ready to test new version of ijavascript package on OpenSuSe if you integrate this solution into tool ;)

[n-riesco]

@drom The way I understand this issue is that Jupyter's packages in OpenSuse are missing the entry point for jupyter kernelspec. If I were you, I'd try to contact the maintainers and make them aware of this issue.

On IJavascript's side, I could try to implement jupyter kernelspec install in pure Javascript, but this is fraught with difficulties, because different versions and installations of IPython and Jupyter use different locations (see https://github.com/nteract/kernelspecs for an attempt to do this for Jupyter).

At the end of the day, it all boils down to the fact that the documented way to install a kernel spec is jupyter kernelspec install.

[drom]

@n-riesco Thank you very much for your help. Your solution totally works for me. I will try contacting OpenSuse Tumbleweed maintainers. And thank you for the great package. Will try using it in practice now!

[drom]

https://bugzilla.opensuse.org/show_bug.cgi?id=1072037

[rgbkrk]

You really only need one function in JS from jupyter-paths for figuring out where to install a kernel. While "finding" kernelspecs available is fraught with mostly python-centric issues, installing one is not. Kernelspecs belong in the data directories, as specified in the common directories and files section of the jupyter docs. There's both an acknowledged collection of user data directories and system data directories, and the order here is the precedence (highest to least).

> require('jupyter-paths').dataDirs()
[ '/Users/kylek/Library/Jupyter',
  '/usr/local/share/jupyter',
  '/usr/share/jupyter' ]

If you run that on a Linux system, you'll see the appropriate results for your system. If not, we can fix it. Within a chosen one of those directories (let's say someDataDir), a new directory should be created with kernelName inside of 'kernels':

var kernelDir = path.join(someDataDir, 'kernels', kernelName);
var kernelspecPath = path.join(kernelDir, 'kernel.json');

That's about it (assuming you handle all the errors in the way users would expect (which you also have to do when shelling out to jupyter).

Note that cocalc uses this in production today and they also come in with fixes for both jupyter-paths and kernelspecs.

[drom]

Thank you @rgbkrk for explanation. Here is my list:

> require('jupyter-paths').dataDirs()
[ '/home/drom/.local/share/jupyter',
  '/usr/local/share/jupyter',
  '/usr/share/jupyter' ]

[n-riesco]

Moved discussion to #137

[benicovdw]

Hi n-riesco: the 9 Dec 2017 post: "Create /usr/bin/jupyter-kernelspec" worked 100% in kali-rolling and julia 0.6.3. Thanks a lot !

[seifoorisara]

Hi, I am using UBUNTU 18.04. I want to use jupyter to use RapidWright, but I can not run these instructions. What is the problem and what can I do? user@parsapc1000:~$ java com.xilinx.rapidwright.util.RapidWright --create_jupyter_kernel Wrote Jupyter Notebook Kernel File: '/home/user/jython27/kernel.json'

You can install the RapidWright (Jython 2.7) kernel by running: $ jupyter kernelspec install /home/user/jython27 and list currently installed kernels with: $ jupyter kernelspec list (base) user@parsapc1000:~$ sudo jupyter kernelspec install /home/user/jython27 [sudo] password for user: Error executing Jupyter command 'kernelspec': [Errno 2] No such file or directory (base) user@parsapc1000:~$

[kikogoinhas]

I had a similar problem. Doing sudo apt-get install jupyter fixed the issue for me.

[dorrajmachai]

@n-riesco This worked on Fedora as well! Thank you! This is really valuable.

[ltsat]

It even worked for the bundled jupyter in sagemath for both ubuntu in termux (via anlinux) and wsl (with ubuntu installed)! Great stuff!