Add support for other kernels

[mkitti]

In principle one should be able to use qtconsole with other Jupyter kernels such as IJulia or IRkernel. Could we add support to allow this to be used with other kernels?

[sofroniewn]

I'm open to this, curious what it would mean from a dependency management standpoint?

i.e. are those kernels already available and just need to be exposed, or do we need to add dependencies? Can you maybe add a link to some documentation about how this is done.

If things are "optional" or "opt in only" then i'd be supportive.

Also sorry for the delay in replying, I didn't have notifications configured for this repo!

[mkitti]

Sure. Basically the kernels are just specified by JSON files. Jupyter has a way to registering these JSON files and listing them.

(base) PS C:\Users\kittisopikulm> jupyter-kernelspec.exe list
Available kernels:
  julia-1.5    C:\Users\kittisopikulm\AppData\Roaming\jupyter\kernels\julia-1.5
  python3      C:\Users\kittisopikulm\.julia\conda\3\share\jupyter\kernels\python3

(base) PS C:\Users\kittisopikulm\.julia\conda\3\share\jupyter\kernels\python3> cat .\kernel.json
{
 "argv": [
  "C:/Users/kittisopikulm/.julia/conda/3\\python.exe",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

(base) PS C:\Users\kittisopikulm\AppData\Roaming\jupyter\kernels\julia-1.5> ls

    Directory: C:\Users\kittisopikulm\AppData\Roaming\jupyter\kernels\julia-1.5

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       12/28/2020     16:14            361 kernel.json
-ar---       12/28/2020     16:14           1310 logo-32x32.png
-ar---       12/28/2020     16:14           2843 logo-64x64.png

(base) PS C:\Users\kittisopikulm\AppData\Roaming\jupyter\kernels\julia-1.5> cat .\kernel.json
{
  "display_name": "Julia 1.5.0",
  "argv": [
    "C:\\Users\\kittisopikulm\\AppData\\Local\\Programs\\Julia 1.5.0\\bin\\julia.exe",
    "-i",
    "--color=yes",
    "--project=@.",
    "C:\\Users\\kittisopikulm\\.julia\\packages\\IJulia\\IDNmS\\src\\kernel.jl",
    "{connection_file}"
  ],
  "language": "julia",
  "env": {},
  "interrupt_mode": "message"
}

[mkitti]

Here's some links to documentation on kernel creation:

https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

[mkitti]

https://jupyter-client.readthedocs.io/en/stable/api/manager.html#jupyter_client.KernelManager

To do the above programmatically:

In [14]: from jupyter_client.kernelspec import KernelSpecManager

In [15]: manager = KernelSpecManager()

In [16]: manager.find_kernel_specs()
Out[16]:
{'julia-1.5': 'C:\\Users\\kittisopikulm\\AppData\\Roaming\\jupyter\\kernels\\julia-1.5',
 'python3': 'C:\\Users\\kittisopikulm\\.julia\\conda\\3\\share\\jupyter\\kernels\\python3'}

In [17]: manager.get_kernel_spec("python3")
Out[17]: <jupyter_client.kernelspec.KernelSpec at 0x1f8055c4310>

In [18]: manager.get_kernel_spec("julia-1.5")
Out[18]: <jupyter_client.kernelspec.KernelSpec at 0x1f8055cfa00>

[mkitti]

The way I would start qtconsole with a Julia kernel is

jupyter qtconsole --kernel "julia-1.5"

That maps to JupyterConsoleApp.kernel_nameUnicode as a config option. https://qtconsole.readthedocs.io/en/stable/config_options.html

I don't think this will add dependencies since you are just using the Jupyter API to locate a JSON file. One would then have to install a kernel separately:

JupyterConsoleApp.kernel_nameUnicode https://github.com/JuliaLang/IJulia.jl

These kernels would be running in a distinct process from Napari and would probably be communicating over ZMQ. While the ability to integrate other kernels into the RichJupyterWidget is likely the first step, more work would likely need to be done to communicate with Napari over ZMQ.

[sofroniewn]

I don't think this will add dependencies since you are just using the Jupyter API to locate a JSON file. One would then have to install a kernel separately:

That's great!

These kernels would be running in a distinct process from Napari and would probably be communicating over ZMQ. While the ability to integrate other kernels into the RichJupyterWidget is likely the first step

Yeah, seems like we just need to provide a way to allow a user to specify which kernel they want? It might be a little tricky with the current API. Maybe it's a persistent setting that changes only on startup? Or maybe we have one widget per kernel? Would you ever want to use more than one kernel at the same time?

more work would likely need to be done to communicate with Napari over ZMQ.

yeah, there have been a couple explorations of things like this in the main repo. also some exploration of communicating with napari using shared memory too.

everything here is pretty experimental still, but if you wanted to make some PRs here that would be appreciated.

The first step is probably just to get this repo being used inside napari instead of the current console which lives inside the main repo. [EDIT: this work has begun https://github.com/napari/napari/pull/2118]