Error with JupyterInspect

[kiyoon]

[lkhphuc]

Hi, thanks for checking out. Can you give me a little more information to reproduce?

Do you have Jupyter kernel running and run JupyterAttach on the buffer? I believe your cursor is on the print function in the screenshot? Below are mine:

P/s: By the way, how did you get that type annotation list[int] on list a?

[kiyoon]

Yes, I followed the setup and installed dependencies. The cursor used to be a or print but regardless it was showing the error.

I also get error on completion

[lkhphuc]

Can you try pip install -U jupyter in the same environment of pynvim package, instead of just pip install -U jupyter_client?

[kiyoon]

For the type inlay hint on python, in short you need to use CoC.nvim.

I still want to use the native LSP so the hacky solution I did was to install CoC but disable everything else on CoC.

https://github.com/kiyoon/dotfiles/blob/e88c76f892952a738d57f529d63897a7b83580c5/nvim/coc-settings.json https://github.com/kiyoon/dotfiles/blob/e88c76f892952a738d57f529d63897a7b83580c5/nvim/lua/kiyoon/lazy.lua#L848-L864

Long answer: https://github.com/lvimuser/lsp-inlayhints.nvim/issues/33
Unfortunately, Microsoft has decided to limit the open source python LSP called pyright and in vscode they use licensed better LSP called pylance. So I can use that plugin for other languages like Lua, but for python I need another solution. CoC improved the pyright server with code actions and inlay hints which is, IMO really an awesome job they pulled off. But I couldn't find a way to use that LSP server with native LSP (maybe there's a way) so I'm just simply using CoC only for inlay hints. Basically running two LSP..:/

[kiyoon]

Improvement idea: consider showing kernel display names instead of kernel id.

[kiyoon]

Can you try pip install -U jupyter in the same environment of pynvim package, instead of just pip install -U jupyter_client?

I just did this, but I still get the same error

[lkhphuc]

Weird, from the cmp traceback it looks like the jupyter server's response does not contain the signature field.

Can you compare with my version:

at 14:52:36  ❯ pip list | grep jupyter
jupyter                       1.0.0
jupyter_client                8.0.3
jupyter-console               6.4.4
jupyter_core                  5.1.3
jupyter-events                0.6.3
jupyter_server                2.1.0
jupyter_server_terminals      0.4.4
jupyterlab-pygments           0.2.2
jupyterlab-widgets            3.0.5

And which method do you use to start the kernel? I tried both jupyter console and jupyter notebook and they still work.

[lkhphuc]

Improvement idea: consider showing kernel display names instead of kernel id.

Love it. Would you mind open a PR for this? Edit: Oh I thought you already make the modification to show this, but it seems this is from a different plugin. I will take a look at it.

[kiyoon]

I think you just need to parse the json files. Should be straightforward.

[kiyoon]

🕙 15:10:52 ❯ pip list | grep jupyter
jupyter                           1.0.0
jupyter_client                    8.0.3
jupyter-console                   6.4.4
jupyter-contrib-core              0.4.2
jupyter-contrib-nbextensions      0.7.0
jupyter_core                      5.1.3
jupyter-events                    0.6.3
jupyter-highlight-selected-word   0.2.0
jupyter-nbextensions-configurator 0.6.1
jupyter_server                    2.1.0
jupyter_server_terminals          0.4.4
jupyterlab                        3.5.2
jupyterlab-pygments               0.2.2
jupyterlab_server                 2.17.0
jupyterlab-widgets                3.0.3

And which method do you use to start the kernel? I tried both jupyter console and jupyter notebook and they still work.

I'm using Jupyter Notebook

[kiyoon]

Another improvement idea: Any plan for async operation? I think you're using the blocking client, so maybe every text change it may block the input. If it can be run in background it will be awesome as well.

[lkhphuc]

If you don't mind, can you clone the repo and install it locally to print out the actual value returned by your kernel? e.g in file rplugin/python3/jupyter_kernel/__init__.py

• line 97 add: self.vim.out_write("Jupyter kernel completiop reply: {}\n".format(reply))
• line 130: self.vim.out_write("Jupyter kernel inspect reply: {}\n".format(reply['content']))
• Then run UpdateRemotePlugin again.

Thank you.

[lkhphuc]

Another improvement idea: Any plan for async operation? I think you're using the blocking client, so maybe every text change it may block the input. If it can be run in background it will be awesome as well.

Yes definitely. I listed it in the future enhance issue #1 since I just want to publish this out first to see if there is anyone interested.

[kiyoon]

Jupyter kernel inspect reply: {'status': 'ok', 'data': {'text/plain': '\x1b[0;31mType:\x1b[0m        DataFra
me\n\x1b[0;31mString form:\x1b[0m\nclass_key        AP    TP  \\\n           0         sphere_slide before s
pher <...> 5                  2582\n           300                 687                  1694\n           \n
          [301 rows x 5 columns]\n\x1b[0;31mLength:\x1b[0m      301\n\x1b[0;31mFile:\x1b[0m        ~/.local/
lib/python3.8/site-packages/pandas/core/frame.py\n\x1b[0;31mDocstring:\x1b[0m  \nTwo-dimensional, size-mutab
le, potentially heterogeneous tabular data.\n\nData structure also contains labeled axes (rows and columns).
\nArithmetic operations align on both row and column labels. Can be\nthought of as a dict-like container for
 Series objects. The primary\npandas data structure.\n\nParameters\n----------\ndata : ndarray (structured o
r homogeneous), Iterable, dict, or DataFrame\n    Dict can contain Series, arrays, constants, dataclass or l
ist-like objects. If\n    data is a dict, column order follows insertion-order.\n\n    .. versionchanged:: 0
.25.0\n       If data is a list of dicts, column order follows insertion-order.\n\nindex : Index or array-li
ke\n    Index to use for resulting frame. Will default to RangeIndex if\n    no indexing information part of
 input data and no index provided.\ncolumns : Index or array-like\n    Column labels to use for resulting fr
ame when data does not have them,\n    defaulting to RangeIndex(0, 1, 2, ..., n). If data contains column la
bels,\n    will perform column selection instead.\ndtype : dtype, default None\n    Data type to force. Only
 a single dtype is allowed. If None, infer.\ncopy : bool or None, default None\n    Copy data from inputs.\n
    For dict data, the default of None behaves like ``copy=True``.  For DataFrame\n    or 2d ndarray input,
the default of None behaves like ``copy=False``.\n\n    .. versionchanged:: 1.3.0\n\nSee Also\n--------\nDat
aFrame.from_records : Constructor from tuples, also record arrays.\nDataFrame.from_dict : From dicts of Seri
es, arrays, or dicts.\nread_csv : Read a comma-separated values (csv) file into DataFrame.\nread_table : Rea
d general delimited file into DataFrame.\nread_clipboard : Read text from clipboard into DataFrame.\n\nExamp
les\n--------\nConstructing DataFrame from a dictionary.\n\n>>> d = {\'col1\': [1, 2], \'col2\': [3, 4]}\n>>
> df = pd.DataFrame(data=d)\n>>> df\n   col1  col2\n0     1     3\n1     2     4\n\nNotice that the inferred
 dtype is int64.\n\n>>> df.dtypes\ncol1    int64\ncol2    int64\ndtype: object\n\nTo enforce a single dtype:
\n\n>>> df = pd.DataFrame(data=d, dtype=np.int8)\n>>> df.dtypes\ncol1    int8\ncol2    int8\ndtype: object\n
\nConstructing DataFrame from numpy ndarray:\n\n>>> df2 = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8
, 9]]),\n...                    columns=[\'a\', \'b\', \'c\'])\n>>> df2\n   a  b  c\n0  1  2  3\n1  4  5  6\
n2  7  8  9\n\nConstructing DataFrame from a numpy ndarray that has labeled columns:\n\n>>> data = np.array(
[(1, 2, 3), (4, 5, 6), (7, 8, 9)],\n...                 dtype=[("a", "i4"), ("b", "i4"), ("c", "i4")])\n>>>
df3 = pd.DataFrame(data, columns=[\'c\', \'a\'])\n...\n>>> df3\n   c  a\n0  3  1\n1  6  4\n2  9  7\n\nConstr
ucting DataFrame from dataclass:\n\n>>> from dataclasses import make_dataclass\n>>> Point = make_dataclass("
Point", [("x", int), ("y", int)])\n>>> pd.DataFrame([Point(0, 0), Point(0, 3), Point(2, 3)])\n   x  y\n0  0
 0\n1  0  3\n2  2  3\n'}, 'metadata': {}, 'found': True}
Error executing Lua callback: /home/kiyoon/.local/share/nvim/runtime/lua/vim/lsp/util.lua:1365: 'replacement
 string' item contains newlines
stack traceback:
        [C]: in function 'nvim_buf_set_lines'
        /home/kiyoon/.local/share/nvim/runtime/lua/vim/lsp/util.lua:1365: in function 'stylize_markdown'
        /home/kiyoon/.local/share/nvim/runtime/lua/vim/lsp/util.lua:1623: in function 'open_floating_preview
'
        ...vim/lazy/jupyter-kernel.nvim/lua/jupyter_kernel/init.lua:49: in function <...vim/lazy/jupyter-ker
nel.nvim/lua/jupyter_kernel/init.lua:16>

Jupyter kernel completiop reply: {'matches': [...: 'csv', 'type': 'instance'}]}, 'status': 'ok'}
Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: TextChangedI Autocommands for "*"..function JupyterComplete[1]..remote#define#
request, line 2: Vim(let):Error invoking '/home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplugin/py
thon3/jupyter_kernel:function:JupyterComplete' on channel 6 (python3-rplugin-host):
error caught in request handler '/home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplugin/python3/jup
yter_kernel:function:JupyterComplete [[None]]':
Traceback (most recent call last):
  File "/home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplugin/python3/jupyter_kernel/__init__.py",
 line 103, in complete
    matches = [
  File "/home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplugin/python3/jupyter_kernel/__init__.py",
 line 108, in <listcomp>
    "value": f"```python\n{m['signature']}\n```",
KeyError: 'signature'
stack traceback:
        [C]: in function 'JupyterComplete'
        ...nvim/lazy/jupyter-kernel.nvim/lua/jupyter_kernel/cmp.lua:22: in function 'complete'
        ...iyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:325: in function 'complete'
        .../kiyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:289: in function 'complete'
        .../kiyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
        .../kiyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:219: in function 'autoindent'
        .../kiyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
        .../kiyoon/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:313: in function 'callback'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/laz
y/nvim-cmp/lua/cmp/utils/autocmd.lua:22>

[kiyoon]

Thanks for looking into this. I'm excited to try this out soon. I like that the plugin is very minimal and is compatible with any Jupyter plugins.

[lkhphuc]

Thank for the log. It seems to be 2 different issues.

For completion, this unfortunately does not show the full log.

Jupyter kernel completiop reply: {'matches': [...: 'csv', 'type': 'instance'}]}, 'status': 'ok'}

I have pushed a test file contains an expected answer from the kernel here https://github.com/lkhphuc/jupyter-kernel.nvim/blob/main/rplugin/python3/test.py . Can you check again if this matches the reply you received? I can't file any standard documentation on the jupyter kernel output so I basically just look at this example from what my kernel returns.

For the inspection, the kernel reply looks fine to me but the error stems from built-in function vim.lsp.util.open_floating_preview. I have also pushed a changed to take extra care of the new line, which make no different on my system (stable and nightly neovim).

Can you update and check again?

[kiyoon]

Error executing Lua callback: .../share/nvim/lazy/lazy.nvim/lua/lazy/core/handler/cmd.lua:32: Vim:Error exec
uting Lua callback: Vim:E117: Unknown function: JupyterKernels
stack traceback:
        [C]: in function 'JupyterKernels'
        ...vim/lazy/jupyter-kernel.nvim/lua/jupyter_kernel/init.lua:4: in function <...vim/lazy/jupyter-kern
el.nvim/lua/jupyter_kernel/init.lua:3>
        [C]: in function 'cmd'
        .../share/nvim/lazy/lazy.nvim/lua/lazy/core/handler/cmd.lua:32: in function <.../share/nvim/lazy/laz
y.nvim/lua/lazy/core/handler/cmd.lua:16>
stack traceback:
        [C]: in function 'cmd'
        .../share/nvim/lazy/lazy.nvim/lua/lazy/core/handler/cmd.lua:32: in function <.../share/nvim/lazy/laz
y.nvim/lua/lazy/core/handler/cmd.lua:16>

Oops, I can't attach to the kernel now

[kiyoon]

Encountered IndentationError loading plugin at /home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplug
in/python3/jupyter_kernel: unexpected indent (__init__.py, line 132)
Traceback (most recent call last):
  File "/home/kiyoon/.local/lib/python3.10/site-packages/pynvim/plugin/host.py", line 165, in _load
    module = imp.load_module(name, file, pathname, descr)
  File "/usr/lib/python3.10/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/usr/lib/python3.10/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 719, in _load
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rplugin/python3/jupyter_kernel/__init__.py",
 line 132
    reply['content']))
IndentationError: unexpected indent
Encountered ModuleNotFoundError loading plugin at /home/kiyoon/.local/share/nvim/lazy/jupyter-kernel.nvim/rp
lugin/python3/test.py: No module named 'jupyter_kernel'
Traceback (most recent call last):
  File "/home/kiyoon/.local/lib/python3.10/site-packages/pynvim/plugin/host.py", line 165, in _load
    module = imp.load_module(name, file, pathname, descr)
  File "/usr/lib/python3.10/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.10/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 719, in _load
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
ModuleNotFoundError: No module named 'jupyter_kernel'

:UpdateRemotePlugins throws this error

[lkhphuc]

Very sorry, can you pull again.

[kiyoon]

I still see the same error Unknown function JupyterKernels
UpdateRemotePlugins is working ok

[lkhphuc]

Hmm.. Have you restart neovim after you update remote plugin?

[WhiteBlackGoose]

Hitting the same issue as OP

Jupyter kernel inspect reply: {'status': 'ok', 'data': {'text/plain': "\x1b[0;3
1mType:\x1b[0m        str\n\x1b[0;31mString form:\x1b[0m aaa\n\x1b[0;31mLength:
\x1b[0m      3\n\x1b[0;31mDocstring:\x1b[0m  \nstr(object='') -> str\nstr(bytes
_or_buffer[, encoding[, errors]]) -> str\n\nCreate a new string object from the
 given object. If encoding or\nerrors is specified, then the object must expose
 a data buffer\nthat will be decoded using the given encoding and error handler
.\nOtherwise, returns the result of object.__str__() (if defined)\nor repr(obje
ct).\nencoding defaults to sys.getdefaultencoding().\nerrors defaults to 'stric
t'.\n"}, 'metadata': {}, 'found': True}

Error executing Lua callback: ...-unwrapped-0.8.1/share/nvim/runtime/lua/vim/ls
p/util.lua:1359: String cannot contain newlines
stack traceback:
        [C]: in function 'nvim_buf_set_lines'
        ...-unwrapped-0.8.1/share/nvim/runtime/lua/vim/lsp/util.lua:1359: in fu
nction 'stylize_markdown'
        ...-unwrapped-0.8.1/share/nvim/runtime/lua/vim/lsp/util.lua:1617: in fu
nction 'open_floating_preview'
        ...vim/lazy/jupyter-kernel.nvim/lua/jupyter_kernel/init.lua:50: in func
tion <...vim/lazy/jupyter-kernel.nvim/lua/jupyter_kernel/init.lua:16>
Press ENTER or type command to continue

I'm on commit 74b7aa5e9c465b966d5df57cb2b84c708cbcf6a3.

Updated to master, hitting the identical issue as the OP

[kiyoon]

OK fixed by restarting.
How do I run the test?

<kibuntu>jupyter-kernel.nvim/rplugin/python3 (1d105707) 📦 🐍 v3.10.6  took 18s
🕙 18:30:23 ❯ python test.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

[lkhphuc]

Hi @kiyoon and @WhiteBlackGoose , can you pull the lastest commit and check again?

I think I've fixed the issue about vim.lsp.util.open_floating_preview cannot display string with new line. Turn out I cannot use [[\n]] in lua but I should use "\n". Even weirder is that both my Mac and my remote Linux server display the info from kernel just fine on multiple neovim version so I didn't notice this earlier. I was only able to reproduce with this command lua vim.lsp.util.open_floating_preview({"first line \n", "second line"}).

@kiyoon For the completion and JupyterKernel not found error, I still have not been able to reproduce that unfortunately.

[WhiteBlackGoose]

Aye, preview works! :heart:

[kiyoon]

Yes, the preview works and it's so good! The completion doesn't work but it also doesn't throw the error either.

[kiyoon]

Let me know what I can do to help reproduce.

[WhiteBlackGoose]

Haven't figured about autocompletion myself, but I don't use nvim-cmp. I added nvim-cmp plugin and config

local cmp = require('cmp')

cmp.setup({
    sources = { name = "jupyter" },
    mapping = { ['<C-Space>'] = cmp.mapping.complete() }
})

but getting no completion :(.