`jupyter kernelspec list --json` returning invalid JSON

[EmmanuelCharpentier]

Setup :

• This is GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37, cairo version 1.16.0) of 2023-03-14, modified by Debian
• from packages-list-package : jupyter 20230214.215 (installed from MELPA).

In a fresh emacs session, M-x jupyter-run-repl fails with Invalid number format: 5. In the *Messages* buffer :

Loading /etc/emacs/site-start.d/00debian.el (source)...done
Loading /etc/emacs/site-start.d/50asymptote.el (source)...done
Loading /etc/emacs/site-start.d/50autoconf.el (source)...done
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...
Loading debian-ispell (native compiled elisp)...
Loading /var/cache/dictionaries-common/emacsen-ispell-default.el (source)...done
Loading debian-ispell (native compiled elisp)...done
Loading /var/cache/dictionaries-common/emacsen-ispell-dicts.el (source)...done
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...done
Loading /etc/emacs/site-start.d/50gnuserv.el (source)...done
Loading /etc/emacs/site-start.d/50latex-cjk-common.el (source)...
Loading /usr/share/emacs/site-lisp/latex-cjk-common/cjk-enc.el (source)...done
Loading /etc/emacs/site-start.d/50latex-cjk-common.el (source)...done
Loading /etc/emacs/site-start.d/50latex-cjk-thai.el (source)...done
Loading /etc/emacs/site-start.d/50latexmk.el (source)...done
Loading /etc/emacs/site-start.d/50lilypond-data.el (source)...done
Loading /etc/emacs/site-start.d/50noweb.el (source)...done
Loading /etc/emacs/site-start.d/50texlive-lang-english.el (source)...done
Loading ispell (native compiled elisp)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
json-read: Invalid number format: 5

This setup (more or less) worked two days ago with the ir kernel (minor issue wit vectors display format) and was nonfunctional with the Wolfram Engine kernel (no output display, see #223).

Results of running M-! jupyter kernelspec list :

Available kernels:
  julia-1.5              /home/charpent/.local/share/jupyter/kernels/julia-1.5
  wolframlanguage13.2    /home/charpent/.local/share/jupyter/kernels/wolframlanguage13.2
  ir                     /usr/local/share/jupyter/kernels/ir
  octave                 /usr/local/share/jupyter/kernels/octave
  python3                /usr/local/share/jupyter/kernels/python3
  sagemath               /usr/local/share/jupyter/kernels/sagemath

[dwinters42]

I had the same problem (OSX 13.3.1, Emacs 29.0.60, python installed with home-brew), updating all python packages using pip with

pip3.11 freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3.11 install -U

and updating my emacs packages solved the problem. Might be worth a try for you ...

[EmmanuelCharpentier]

Thanks bor the hint. Bit I can't use it : I'm using Debian testing, whose pip is managed by Debian. pip3.11 fails to install or upgrade packages, with the following error message :

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Any hint ?

[md-zengo]

I'm experiencing the same issue - and updating Python and Emacs packages is not solving it.

[EmmanuelCharpentier]

Thanks bor the hint. Bit I can't use it : I'm using Debian testing, whose pip is managed by Debian. pip3.11 fails to install or upgrade packages, with the following error message :

I solved this. But even after this cleanup and a re-installation of emacs-jupyter, the problem persists...

[md-zengo]

I don't understand - how did you solve it @EmmanuelCharpentier ?

[guseggert]

Seems like this is caused by some new warning text in the output, e.g.:

$ jupyter kernelspec list
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Available kernels:
  python3    /Users/gus/.../venv/share/jupyter/kernels/python3

I can workaround this with (setenv "PYDEVD_DISABLE_FILE_VALIDATION" "1"). As the warning says, this probably messes up debugging, but I don't really use that with this. There's probably a better way to pass -Xfrozen_modules=off to the kernel but I haven't dug into that.

[EmmanuelCharpentier]

I don't understand - how did you solve it @EmmanuelCharpentier ?

I more or less reinstalled all Python-related in my system and re-added non-packaged pip packages via pipx. Not too much of a task thanks to a bit of bash plumbing ;-)...

[EmmanuelCharpentier]

@guseggert :+1: !

(setenv "PYDEVD_DISABLE_FILE_VALIDATION" "1")

Thanks ! This unborked my Juyter kernels.

Nice find ! How did you came by this ?

[guibor]

Thanks! This worked.

[guseggert]

Nice find ! How did you came by this ?

You mentioned jupyter kernelspec list so I ran that in my virtualenv and noticed the warning message which triggered my spidey sense, given the error was a parsing error.

[goodvibrations32]

@guseggert

I can workaround this with (setenv "PYDEVD_DISABLE_FILE_VALIDATION" "1").

This helped me also as well!!! Thanks a lot! (I'm depending on jupyter notebooks for my bachelor thesis so you saved me from a stroke!! thanks again!)

[nnicandro]

Related is #423.

[samleeney]

(setenv "PYDEVD_DISABLE_FILE_VALIDATION" "1"

THANK YOU

[dragove]

These messages are provided by debugpy, uninstall debugpy and everything will work.