tab completion not working in jupyterlite when using xeus-python kernel

[lesteve]

To reproduce:

• go to https://xeus-python-kernel.readthedocs.io/en/latest/
• type pr<TAB> no completions is shown

Expected behaviour: completions are shown showing print for example

Observed behaviour: no completion is shown.

I originally observed the behaviour when using jupyterlite build --XeusPython.packages locally

I tried both in Firefox and Chromium, same behaviour.

Pyolite tab-completion seems to work fine e.g. on https://numpy.org/

Output from the browser console:

ERROR: received bad message: NotImplementedError: Python version 3.10 is currently not supported. xpython_wasm.js:9:53168
<empty string> xpython_wasm.js:9:53168
At: xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/parso/grammar.py(264): load_grammar xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/environment.py(37): get_grammar xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/cache.py(112): wrapper xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/inference/__init__.py(91): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/__init__.py(119): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/__init__.py(729): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(1397): _jedi_matches xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(2054): _complete xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(1886): _completions xpython_wasm.js:9:53168
<empty string> xpython_wasm.js:9:53168
Message type: complete_request xpython_wasm.js:9:53168

Here is a screenshot in case that helps

[martinRenou]

Thank you for opening an issue!

We should probably check if we're using the latest version of parso/jedi.

[lesteve]

I think this is using the latest version of parso/jedi.

Inside the Jupyterlite from https://xeus-python-kernel.readthedocs.io/en/latest/,

import jedi; print(f"{jedi.__version__=}")
import parso; print(f"{parso.__version__=}")

​

jedi.__version__='0.18.1'
parso.__version__='0.8.3'

which match the latest versions from https://pypi.org/project/jedi/#history and https://pypi.org/project/parso/#history

[jtpio]

For reference this still seems to be the case with the latest versions of jupyterlite-xeus-python and jedi / parso:

[jtpio]

Just checked with the latest jedi 0.19 and there is still the same issue:

[martinRenou]

Using the https://xeus-python-kernel.readthedocs.io deployment I see completion working:

Although it's true that pr<tab> does not show print as an option for some reason:

[jtpio]

This is what I get with the following code snippet and pressing TAB:

import sys
sys.

https://github.com/jupyterlite/xeus-python-kernel/assets/591645/a9990b7c-1fdf-4b56-b638-1503cbd1d52a

[martinRenou]

I can't make sense of the behavior difference between classic xeus-python and jupyterlite-xeus-python as they both go through the same code path for the code completion.

One thing I thought about was that we may want to disable using jedi in the code completion in IPython. For pyodide-kernel it has been disabled in https://github.com/jupyterlite/pyodide-kernel/commit/e2e6819614ebe658a48abac4bcce8e66384da01b.

[martinRenou]

Yep, that seems to do the trick:

We can probably disable jedi in xeus-python by default. I would not be against this.

[jtpio]

Also what does ipykernel do to provide "nicer" completions out of the box compared to xeus python?

ipykernel

xeus-python

[martinRenou]

We're probably missing some metadata in the completion answer.

[martinRenou]

Indeed, when using jedi, ipykernel fills up some metadata:

https://github.com/ipython/ipykernel/blob/c8517461f434a72fb4e30fc03b5949aca4c5af52/ipykernel/ipkernel.py#L540C26-L540C48

[martinRenou]

And JupyterLab uses this information here https://github.com/jupyterlab/jupyterlab/blob/bddc378b72e8ee78be8b6e2ffb2b527a75f0471a/packages/completer/src/default/kernelprovider.ts#L61

We may want to explore implementing the same when not using Jedi. It might be possible, depending on what the IPython's Interactive Shell Completer gives us as information.

[jtpio]

I can't make sense of the behavior difference between classic xeus-python and jupyterlite-xeus-python as they both go through the same code path for the code completion.

Could there also be a way to know why jedi does not behave as expected with jupyterlite-xeus-python?

Wondering if we could keep it enabled in the long run instead of disabling it that would help reduce the number of differences between "normal Jupyter" and JupyterLite.