jupyter-session-with-random-ports: ‘jupyter kernel‘ failed to show connection file path

[fountainer]

I'm using doom-emacs (29.1 ) on mac. Recently, I want to upgrade to the newest version of emacs-jupyter (0a92c0c) from the version before merging the next branch. However, executing jupyter-execute-repl gives the following error.

Debugger entered--Lisp error: (error "‘jupyter kernel‘ failed to show connection file path")
  error("`jupyter kernel` failed to show connection file path")
  jupyter-session-with-random-ports()
  #f(compiled-function (kernel) "Ensure KERNEL has a non-nil SESSION slot.\nA `jupyter-session' with random port numbers for the channels and\na newly generated message signing key will be set as the value of\nKERNEL's SESSION slot if it is nil." #<bytecode -0x15dcbed52308f068>)(#<jupyter-kernel ir>)
  apply(#f(compiled-function (kernel) "Ensure KERNEL has a non-nil SESSION slot.\nA `jupyter-session' with random port numbers for the channels and\na newly generated message signing key will be set as the value of\nKERNEL's SESSION slot if it is nil." #<bytecode -0x15dcbed52308f068>) #<jupyter-kernel ir>)
  #f(compiled-function (&rest args) #<bytecode -0x1f7b3412dabefcc8>)(#<jupyter-kernel ir>)
  apply(#f(compiled-function (&rest args) #<bytecode -0x1f7b3412dabefcc8>) #<jupyter-kernel ir> nil)
  jupyter-launch(#<jupyter-kernel ir>)
  #f(compiled-function (kernel) #<bytecode -0x1e07c58939debbf0>)(#<jupyter-kernel ir>)
  apply(#f(compiled-function (kernel) #<bytecode -0x1e07c58939debbf0>) #<jupyter-kernel ir> nil)
  jupyter-zmq-io(#<jupyter-kernel ir>)
  #f(compiled-function (kernel) "Return an I/O connection to KERNEL's session." #<bytecode 0x19daa9c510a84b3d>)(#<jupyter-kernel ir>)
  apply(#f(compiled-function (kernel) "Return an I/O connection to KERNEL's session." #<bytecode 0x19daa9c510a84b3d>) #<jupyter-kernel ir>)
  #f(compiled-function (&rest args) #<bytecode -0x16a95ca5bd3b346d>)()
  #f(compiled-function (cl--cnm thing) #<bytecode -0xed21745ea97e5f4>)(#f(compiled-function (&rest args) #<bytecode -0x16a95ca5bd3b346d>) #<jupyter-kernel ir>)
  apply(#f(compiled-function (cl--cnm thing) #<bytecode -0xed21745ea97e5f4>) #f(compiled-function (&rest args) #<bytecode -0x16a95ca5bd3b346d>) #<jupyter-kernel ir>)
  #f(compiled-function (thing) "Cache the I/O object of THING in `jupyter-io-cache'." #<bytecode 0x19f23bef1dd9ad45>)(#<jupyter-kernel ir>)
  apply(#f(compiled-function (thing) "Cache the I/O object of THING in `jupyter-io-cache'." #<bytecode 0x19f23bef1dd9ad45>) #<jupyter-kernel ir> nil)
  jupyter-io(#<jupyter-kernel ir>)
  #f(compiled-function (kernel &optional client-class) #<bytecode -0xf477e7e4a1fe9bf>)(#<jupyter-kernel ir> jupyter-repl-client)
  apply(#f(compiled-function (kernel &optional client-class) #<bytecode -0xf477e7e4a1fe9bf>) #<jupyter-kernel ir> jupyter-repl-client)
  jupyter-client(#<jupyter-kernel ir> jupyter-repl-client)
  jupyter-run-repl("ir" nil t nil t)
  funcall-interactively(jupyter-run-repl "ir" nil t nil t)
  command-execute(jupyter-run-repl record)
  execute-extended-command(nil "jupyter-run-repl" nil)
  funcall-interactively(execute-extended-command nil "jupyter-run-repl" nil)
  command-execute(execute-extended-command)

[fast-90]

I am having the same issue on MacOS, emacs-plus (29.1) on MacOS as well.

[fakeGenuis]

Same in doom and Arch linux. Run jupyter kernel in shell returns "RuntimeError: This event loop is already running". Solved by downgrade python-jupyter-core to 5.5.1-1. (it was flagged out-of-date in arch package repository, maybe next upgrade will fix)

[mk0417]

I had the same problem after I upgraded jupyter-core to 5.6.0 recently. The version of 5.5.1 works.

[MasonProtter]

I'm also hitting this.

[dangom]

This happens to me as well, but retrying jupyter-run-repl will eventually work. It used to always work before, so definitely something changed with jupyter-core. I'm on the latest master.

[delnatan]

I had this same problem. I'm running Emacs 29.2 on MacOS (installed via homebrew tap emacs-plus@29; https://github.com/d12frosted/homebrew-emacs-plus). I installed emacs-jupyter using straight.el from the github repo.

Now, strangely enough, when I start emacs with --debug-init. I can run jupyter-repl-run just fine!

I'm running the jupyter kernel installed in a micromamba environment (using micromamba.el).

I'm kind of new to emacs, what could be the difference when doing --debug-init versus launching Emacs via the link from the '/Applications' folder?

[delnatan]

ok, I don't need to pass "--debug-init", simply starting emacs from my terminal allows me to run jupyter-run-repl just fine without a 'jupyter-session-with-random-ports: ‘jupyter kernel‘ failed to show connection file path' error. Looking at my $PATH, the only difference between emacs exec-path and my $PATH are: var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin

[akirakyle]

I am also seeing the same issue, but only when trying to run on remote hosts via tram. I was able to debug this and fix it for me locally and my fix may help others here debug what's going on in their setups.

I am specifying :session /ssh:akyle@myserver.com:julia while on myserver I have the output of jupyter --paths as

config:
    /home/akyle/.jupyter
    /home/akyle/.local/etc/jupyter
    /home/akyle/.julia/conda/3/x86_64/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/akyle/.local/share/jupyter
    /home/akyle/.julia/conda/3/x86_64/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/akyle/.local/share/jupyter/runtime

The python binary that jupyter is using is in /home/akyle/.julia/conda/3/x86_64/bin/python3 however jupyter-locate-python returns /bin/python3 which is what jupyter-session-with-random-ports starts and then tries run from jupyter_client.kernelapp import main; main() in. However since its the wrong python, the jupyter import fails. I think this may be a generic failure mechanism whever jupyter-locate-python finds a python binary before it actually finds the correct one jupyter is using. This is because the first path it searches upward from is /home/akyle/.local/share/jupyter rather than /home/akyle/.julia/conda/3/x86_64/share/jupyter. I was able to fix this for myself by modifying changing the line in jupyter-locate-python from

                    for spath = (expand-file-name program dir)

to

                    for spath = (expand-file-name-remote program dir)

where the definition of expand-file-name-remote is

(defun expand-file-name-remote (file dir)
  (let* ((dir-local (file-local-name dir))
         (file (expand-file-name file dir-local))
         (dir-remote (file-remote-p dir)))
    (concat dir-remote file)))

however this actually only works for the wrong reasons that locate-dominating-file will search

/ssh:akyle@myserver.com:~/.local/share/jupyter
/ssh:akyle@myserver.com:~/.local/share/
/ssh:akyle@myserver.com:~/.local/
/ssh:akyle@myserver.com:~/
/ssh:akyle@myserver.com:

but expand-file-name-remote treats /ssh:akyle@myserver.com:~/ and /ssh:akyle@myserver.com: the same as the same remote directories as that's how tramp treats them in file-local-name.

Ideally there would be a more robust way for jupyter-locate-python to get the correct path. Perhaps with jupyter console and then import sys; sys.executable?

[samJcrawford]

What is the reason for using jupyter-locate-python here in the first place? I have just replaced the call to jupyter-locate-python with a string variable (defaulting to "python3"), and haven't seen any issues yet.